Skip to main content

Upgrading

Upgrading a Helm-installed Avatar deployment is a single helm upgrade against the same values file you installed with. This guide covers the safe procedure, and the one place where Kubernetes differs from the Docker path in a way you need to know about before you start.

Recovery is by re-installing, not by rolling back

helm rollback is untested against this chart. Plan your recovery around re-installing the previous chart version from a database backup — see Rolling back.

Prerequisites​

Before starting the upgrade, ensure you have:

  • kubectl and helm access to the cluster and the release namespace
  • The values file you installed with, up to date with any changes made since
  • The target chart version and the matching image tags from Octopize
  • Network connectivity to pull the new images (or pre-mirrored images for air-gapped deployments):
    • Egress on port 443 to quay.io for the Avatar images and the chart artifact
    • Egress on port 443 to docker.io for supporting images
    • Egress on port 443 to ghcr.io for Authentik images
  • Enough scheduling headroom for a rolling update of the web and Dask deployments

Pre-upgrade checklist​

All items should be verified before proceeding.

  • Current chart version and image tags documented (helm -n avatar list)
  • Release notes for the target version reviewed
  • Full backup of the PostgreSQL database completed
  • Your values file backed up, and its diff against the deployed release reviewed (helm -n avatar get values avatar)
  • Maintenance window scheduled and communicated to users
  • Recovery plan reviewed and understood (see Rolling back)

Check the current version​

helm -n avatar list
kubectl -n avatar get deploy -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.template.spec.containers[0].image}{"\n"}{end}'

Back up the database​

Take a snapshot, or pg_dump against database.host. Back up both the Avatar database (database.name) and Authentik's (authentik.authentik.postgresql.name) — restoring one without the other leaves logins broken.

If you rely on the in-chart SeaweedFS, snapshot its PersistentVolumes as well.

Set the SeaweedFS admin password​

The chart releases after 0.0.97 need a SeaweedFS admin password. Without it, the chart does not render. Before you upgrade, add storage.seaweedfsAdminPassword to the file of generated secrets that you keep, for example avatar-secrets.yaml:

storage:
seaweedfsAdminPassword: "<output of: openssl rand -hex 32>"

To read the password from a Secret of your own instead, see SeaweedFS.

Upgrade to chart 0.1.0 through chart 0.0.102​

Chart 0.1.0 runs Authentik 2026.8, which starts only on a database that already ran Authentik 2026.5. Authentik upgrades its database one release line at a time, and cannot downgrade it.

  • From chart 0.0.101 or older, upgrade to chart 0.0.102 first, which runs Authentik 2026.5. Wait until Authentik is ready on it:

    kubectl -n avatar rollout status deploy/avatar-authentik-server

    Then upgrade to 0.1.0, with the same procedure.

  • A new installation can install chart 0.1.0 directly.

  • If you skip chart 0.0.102, the Authentik server stops at start with Major version skips are not allowed, before it changes the database. Upgrade to chart 0.0.102, let Authentik start, then upgrade to 0.1.0 again.

  • Back up Authentik's database before each of the two upgrades.

Chart 0.1.0 also gives Authentik its base URL, AUTHENTIK_WEB__BASE_URL, from ingress.authentikHost or ingress.host. Authentik requires a base URL from 2026.11.

Restart Authentik after the release that adds sso.trustedProxyCidrs​

The chart release that adds sso.trustedProxyCidrs gives Authentik the networks that it takes X-Forwarded-Proto and X-Forwarded-For from. Its release notes name it. Authentik reads this value only at start, and helm upgrade does not restart it. After you upgrade to that release, and after each change to sso.trustedProxyCidrs, restart the Authentik server:

kubectl -n avatar rollout restart deploy/avatar-authentik-server
kubectl -n avatar rollout status deploy/avatar-authentik-server

Until it restarts, an Authentik server on a pod network outside the Authentik default, for example 100.64.0.0/10 on Scaleway Kapsule, takes every request for plain HTTP, and the login returns to its first screen. See The login returns to its first screen.

Running the upgrade​

Step 1: update the image tags in your values file​

Three values move together on every version bump:

api:
image:
tag: "<new-api-version>"

web:
image:
tag: "<new-web-version>"

dask:
scheduler:
image:
tag: "<new-api-version>" # must equal api.image.tag
worker:
image:
tag: "<new-api-version>" # must equal api.image.tag

If you run the database connector, its image has its own version line: connector.image.tag does not follow api.image.tag. Raise it when the release notes ask you to.

Step 2: preview the change​

Rendering the new chart with your values file catches a missing or contradictory value before anything in the cluster changes:

helm upgrade avatar oci://quay.io/octopize/helm/services-api/avatar \
--version <new-version> \
--namespace avatar \
--values my-values.yaml \
--dry-run

Step 3: upgrade​

helm upgrade avatar oci://quay.io/octopize/helm/services-api/avatar \
--version <new-version> \
--namespace avatar \
--values my-values.yaml

Two things happen automatically as the new pods roll out:

  • Database migrations, run by the API's init container. A failure shows up as the API pod staying in Init, with the old pod still serving traffic.
  • The Authentik bootstrap Job re-runs. It does nothing when the database already exists.

Step 4: monitor the rollout​

kubectl -n avatar get pods -w
kubectl -n avatar rollout status deploy/avatar-api
kubectl -n avatar logs deploy/avatar-api -c init-avatar-api

Step 5: verify the deployment​

Run through the Verification & Troubleshooting checks — at minimum the API health check, the login flow test and a sample avatarization.

Post-upgrade verification checklist​

  • All pods Running, none in a restart loop
  • avatar-authentik-db-init-<new-chart-version> Job Complete
  • Authentik server restarted, if the release adds sso.trustedProxyCidrs or you changed it
  • API health endpoint returns OK with the correct version
  • User login succeeds through Authentik
  • Sample avatarization job completes successfully
  • If the database connector is enabled, its pod is Running and Test succeeds on a demo connection
  • Object storage is accessible, with no SSE-S3 errors
  • PDF report generation works
  • Email sending works

Keep the API at one replica​

Leave api.replicaCount at 1 across the upgrade, and in general. More than one API pod is not currently supported.

Rolling back​

Plan recovery as a fresh install of the previous version:

  1. Restore the PostgreSQL databases from the backup taken before the upgrade. Migrations are not reversible, so a database migrated forward will not work with the older API image.
  2. Re-run helm upgrade with the previous chart version and image tags, from the values file you backed up.
  3. Verify with the Verification & Troubleshooting checks.

Before doing any of that, collect diagnostic information so the issue can be investigated afterwards.

warning

If you are on a managed PostgreSQL where you cannot restore in place, restore to a new database and repoint database.host / database.name rather than upgrading over a database that has already been migrated forward.

Best practices​

  • Test upgrades against a staging cluster first
  • Upgrade during low-usage periods
  • Keep the values file in version control, so the diff between two releases is reviewable
  • Keep the upgrade output and the init-container logs for troubleshooting

Getting help​

If you encounter issues during the upgrade:

  1. Check the logs: the API init container first, then the Authentik bootstrap Job
  2. Verify the configuration: helm -n avatar get values avatar against your values file
  3. Collect diagnostics: follow the diagnostic collection procedure
  4. Contact support: reach out to Octopize support with the specific error messages and the diagnostic files

For emergency recovery assistance, contact Octopize support immediately with your backup files ready.