Module 06 · Real-world delivery · 50 min · You are: Platform team

Expose Argo CD safely, and upgrade it

Argo CD can change everything in the cluster, so its front door matters more than most. A port-forward is fine for one person; a team needs a stable URL, TLS, and accounts that say who did what. And an install nobody upgrades becomes the one nobody dares to touch.

Open this module in the interactive course to track progress and switch tracks.

GitOps loop practised: Committed → Reviewed → Detected → Rendered → Compared → Synced → Healthy → Observed

Concepts

One front door

An Ingress with TLS. Argo CD serves plain HTTP behind it (server.insecure), which is only safe because TLS ends at the ingress and nothing else reaches the Service.

gRPC through an ingress

The CLI speaks gRPC. Through a generic HTTP ingress use argocd login --grpc-web.

No shared admin

Personal accounts (here: local users) or SSO (in production: OIDC). Disable admin once someone else can administer, and delete the initial password Secret.

Upgrades are changes

Read the release and upgrade notes, render old and new, change targetRevision, merge, verify. Patch: routine. Minor: read notes. Major: test first.

Ways to reach Argo CD

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

• platform/argocd/values.yaml (reference): Today's additions: exposure and accounts.
  1. Plain HTTP behind the TLS-terminating ingress L7–10
  2. The shared admin account, switched off L14–15
  3. Logged in but not listed: no access. alice is an admin L37–40
  4. The Ingress, on Traefik's HTTPS entrypoint L49–56
• platform/apps/argocd.yaml (reference): The upgrade is one line.
  1. The chart version you upgrade to today L13

Steps

1. Expose the UI and API through the ingress

Add server.insecure and the Ingress to the values. After the sync, open https://argocd.localtest.me:8443 (the same certificate warning as in module 01: Traefik serves its own default certificate, and you continue the same way). This is a new address: until now you used localhost:8080, which was the port-forward; from here on the UI and the CLI go through the ingress instead. The last command of this step moves the CLI over, and only then can you stop the port-forward.

Existing cluster: set ingressClassName to your class, global.domain to a host you control, and replace the Traefik annotations with your controller's TLS settings.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Add to ~/work/argocd-platform/argocd/values.yaml under configs:, above cm:

  params:
    # TLS ends at the ingress controller (Traefik); Argo CD serves plain HTTP
    # behind it. Only safe because nothing else can reach the server Service.
    server.insecure: true

Add to ~/work/argocd-platform/argocd/values.yaml at the top level (no indentation), above the # No SSO on a laptop comment

server:
  ingress:
    enabled: true
    ingressClassName: traefik
    annotations:
      # Serve on Traefik's HTTPS entrypoint with its default certificate.
      traefik.ingress.kubernetes.io/router.entrypoints: websecure
      traefik.ingress.kubernetes.io/router.tls: "true"

Check your edits. Expect only added lines: the params block and the server: block with the ingress.

git diff

Propose it as a pull request. Put it on its own branch and push it:

git switch -c expose && git add -A && git commit -qm "Argo CD: ingress with TLS"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

Argo CD applies its own change. First tell it to look at Git now (it would find your merge on its own within about 3 minutes; this just skips the wait), then wait until it has finished:

argocd app get argocd --refresh >/dev/null
argocd app wait argocd --sync --health --timeout 300

Knock on the new front door. Expect 200. If you get 000, 404 or 502, Argo CD has not finished applying the change yet: run the two commands above again, wait a few seconds, and retry.

curl -sk -o /dev/null -w '%{http_code}\n' https://argocd.localtest.me:8443/healthz

Switch from the port-forward to the ingress. Until now the CLI logged in to localhost:8080, the port-forward from module 01. From this point every argocd command uses argocd.localtest.me:8443 (on kind that is argocd.localtest.me:8443: your laptop's port 8443 reaches Traefik inside the cluster, and localtest.me always points to your own machine). Log the CLI in again through the ingress. --grpc-web lets the CLI talk through a normal web proxy; --insecure accepts Traefik's self-signed certificate.

argocd login argocd.localtest.me:8443 --grpc-web --insecure --username admin --password "$(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d)"

Now you can stop the port-forward: press Ctrl+C in the terminal where it runs, and close that terminal. If a command later fails with connection refused on localhost:8080, you logged in to the old address: run the login above again.

2. A personal account for the platform team

Declare alice and make her an admin in Git. The password is never in Git: set it with the CLI, as admin, once.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Add to ~/work/argocd-platform/argocd/values.yaml under cm:, as its first line

    accounts.alice: login            # platform team

Add to ~/work/argocd-platform/argocd/values.yaml under configs:, after the last cm: line (the banner), at the same indentation as cm:

  rbac:
    # Logged in but not listed below: see nothing.
    policy.default: ""
    policy.csv: |
      # Platform team: everything.
      g, alice, role:admin

Propose it as a pull request. Put it on its own branch and push it:

git switch -c account-alice && git add -A && git commit -qm "Argo CD: account for alice"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

argocd app get argocd --refresh >/dev/null
argocd app wait argocd --sync --health

Set alice's password. Git only says the account exists; the password is stored in the cluster, never in Git. You set it once, as admin:

argocd account update-password --account alice --current-password "$(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 -d)" --new-password 'choose-a-long-one'

Log in as alice:

argocd login argocd.localtest.me:8443 --grpc-web --insecure --username alice --password 'choose-a-long-one'

Check who you are. Expect Logged In: true and Username: alice:

argocd account get-user-info

3. Switch off the shared admin

Now that alice can administer, disable admin and delete the bootstrap password.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Add to ~/work/argocd-platform/argocd/values.yaml under cm:, at the top, above accounts.alice

    # Module 06: the shared admin account is off. People log in as themselves.
    admin.enabled: false

Propose it as a pull request. Put it on its own branch and push it:

git switch -c no-admin && git add -A && git commit -qm "Argo CD: disable the admin account"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

argocd app get argocd --refresh >/dev/null
argocd app wait argocd --sync --health

Delete the one-time password Secret from the install. It is no longer needed, and a leftover password is a risk:

kubectl -n argocd delete secret argocd-initial-admin-secret

Prove admin is off. This login is meant to fail, with a message that the account is disabled:

argocd login argocd.localtest.me:8443 --grpc-web --insecure --username admin --password x

4. Prepare the upgrade: read, then render both versions

Read the Argo CD release notes between your version and v3.5.6, and the chart's changelog. Then see what actually changes in the cluster: mostly image tags here.

Compare the two versions. helm template builds the manifests for each chart version, and diff shows what differs. grep keeps only the image lines. Expect a few pairs of < (old) and > (new) lines with different version numbers, and nothing else: the upgrade only changes images.

cd ~/work/argocd-platform
diff <(helm template argocd argo/argo-cd --version 10.10.5 -n argocd -f argocd/values.yaml) \
     <(helm template argocd argo/argo-cd --version 10.11.6 -n argocd -f argocd/values.yaml) | grep -E '^[<>] +image:' | sort -u

5. Upgrade Argo CD with a pull request

One line. Argo CD rolls its own components; the UI may blink. Then confirm the new server version.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Change in ~/work/argocd-platform/apps/argocd.yaml

-       targetRevision: 10.10.5
+       targetRevision: 10.11.6

Expect exactly one changed line: the version number.

git diff

Propose it as a pull request. Put it on its own branch and push it:

git switch -c upgrade-argocd && git add -A && git commit -qm "Upgrade Argo CD: chart 10.11.6 (v3.5.6)"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

Argo CD replaces its own pods one by one. Wait until it is healthy again (the UI may blink; this can take a few minutes):

argocd app get argocd --refresh >/dev/null
argocd app wait argocd --sync --health --timeout 600

Confirm the new version. Expect argocd-server: v3.5.6:

argocd version --short | grep argocd-server

Simplified for the course

Traefik serves a self-signed certificate, hence --insecure on the CLI. In production: cert-manager with a real certificate, a private ingress or VPN, and --insecure nowhere.

Local accounts stand in for SSO. In production configure OIDC (or Dex) in the same values file, map identity-provider groups to roles, and keep one local break-glass account whose password is in your vault.

Downgrades across minor versions are not supported. Rolling back an upgrade means reverting the pull request only for patch releases; for minor and major upgrades, test on a throwaway cluster first.

Break it on purpose: Lock yourself out with a commit

A "tidy-up" pull request removes alice from the RBAC policy. The shared admin account is already switched off, so alice was the only administrator.

You practise: recovering from a bad configuration change when neither the UI nor the CLI can help you, and knowing which part of Argo CD needs no login.

1. Break it

cd ~/work/argocd-platform && git switch -q main && git pull -q

Delete from ~/work/argocd-platform/argocd/values.yaml

-       g, alice, role:admin

Propose it as a pull request. Put it on its own branch and push it:

git switch -c tidy-rbac && git add -A && git commit -qm "Tidy RBAC"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

Give Argo CD a minute to apply it:

sleep 60

2. What you see

Reload the UI as alice. Expected: every Application. You get: an empty Applications page. You are logged in, but no policy gives you any rights. The CLI agrees: argocd app get argocd says permission denied.

3. Fix it yourself

Give alice her admin rights back, without kubectl and without switching the admin account back on.

cd ~/work/argocd-platform && git switch -q main && git pull -q

git switch -c undo-tidy-rbac && git revert --no-edit HEAD
git push -qu origin HEAD

gh pr create --fill

gh pr merge --squash --delete-branch

git switch -q main && git pull -q

sleep 60; argocd app get argocd | grep -E 'Sync Status|Health'

Quiz

1. Why is server.insecure: true acceptable here?
   • TLS terminates at the ingress, and only the ingress can reach the Argo CD server
   • It is never acceptable, because Argo CD must always terminate TLS itself on its own port
   • It is fine only because the cluster is a laptop; on a real cluster it would have to be false
2. Where does alice's password live?
   • In values.yaml, hashed with bcrypt, so a reviewer can see exactly who has which password before it is merged
   • In the argocd-secret Secret, set with the CLI; Git declares only that the account exists
   • In the root Application, which stores every account's password next to its role binding so both change together
3. What is the upgrade, technically?
   • A helm upgrade run from your laptop against the cluster, with the new chart version
   • Deleting Argo CD and reinstalling the newer chart, then re-adding every Application by hand
   • A pull request changing targetRevision of the argocd Application

Field note

Teams that upgrade every month have boring upgrades. Teams that wait a year read three sets of breaking changes at once. Put a recurring reminder on the calendar, or let Renovate open the pull request for you.

Report a problem with this module