PlatformEngr® Argo CD course

Module 04 · Foundations · 45 min · You are: Platform team

App of apps: everything declarative, Argo CD included

In this module

  • You learn: The app-of-apps pattern: one root Application that creates all the others from a folder in Git. How Argo CD can manage its own installation. How branch protection and a CI check make pull requests the only way in.
  • You do: Add a validation check and protect main. Open a pull request that adds the root Application and an Application for Argo CD itself. Apply the root once. Then change Argo CD\'s settings through a pull request.
  • You end with: One root Application owns argocd and checkout-dev. Adding or removing a file in apps/ adds or removes an Application. main accepts changes only through a pull request that passes validate. kubectl apply for Applications is over.

Until now Applications were applied by hand, and Argo CD itself was a Helm release you could change from your laptop. After this module there is exactly one manual step, ever: applying the root Application. Everything else, including upgrading Argo CD, is a file in Git that went through review.

Open this module in the interactive course to track progress and switch tracks.

GitOps loop practised: Committed → Reviewed → Detected → Rendered → Compared → Synced → Healthy → Observed

Concepts

App of apps
The root Application's source is the apps/ folder, which contains Application manifests. Add a file, get an Application; delete it, the Application goes.
Argo CD manages itself
An Application named argocd points at the same chart and values you installed with. Argo CD adopts the resources Helm created; from then on, changes to Argo CD are commits.
The one manual step
kubectl apply -f bootstrap/root.yaml. It is in Git too, outside apps/, so it cannot delete itself.
Finalizers
resources-finalizer.argocd.argoproj.io on an Application means: deleting it deletes what it deployed. Without it, deleting the Application leaves the workload running.

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

Steps

1. Guardrails first: validation and branch protection

Add the validate workflow and script, push directly, then in branch settings protect main: require a pull request and the validate status check (it appears after the first run). You cannot approve your own pull requests, so do not require approvals while you work alone.

Copy the check from the reference. It is a script plus a GitHub workflow that runs it on every pull request:

cd ~/work/argocd-platform
mkdir -p .github/workflows scripts
cp ~/work/argocd-gitops/platform/.github/workflows/validate.yaml .github/workflows/
cp ~/work/argocd-gitops/platform/scripts/validate.sh scripts/

Run the check yourself first. It reads every manifest in the repo and complains about plain Secrets or Applications without a project. Expect it to finish without errors:

scripts/validate.sh

Commit and push straight to main:

git add -A && git commit -qm "CI: validate manifests" && git push -q

2. Propose the root app and Argo CD managing itself

Copy both from the reference, point them at your repo, keep the default project and keep the chart at the version you installed: adopting is not the moment to upgrade.

cd ~/work/argocd-platform && git switch -q main && git pull -q
cp ~/work/argocd-gitops/platform/bootstrap/root.yaml bootstrap/
cp ~/work/argocd-gitops/platform/apps/argocd.yaml apps/
perl -pi -e 's/YOUR_GITHUB_USER/YOUR_GITHUB_USER/g' bootstrap/root.yaml apps/argocd.yaml     # your GitHub user instead of the placeholder

Change in ~/work/argocd-platform/bootstrap/root.yaml, ~/work/argocd-platform/apps/argocd.yaml
In both files: the reference uses a project you have not created yet.

-   project: platform
+   project: default

Change in ~/work/argocd-platform/apps/argocd.yaml
The version you installed: adopt first, upgrade later.

-       targetRevision: 10.11.6
+       targetRevision: 10.10.5

Run the check on your edits:

scripts/validate.sh

Propose the change as a pull request. First put it on its own branch and push it:

git switch -c root-app && git add -A && git commit -qm "Root app; Argo CD manages itself"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch). In a team, a reviewer approves here; on your own repo you merge it yourself.

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

3. Apply the root app: the last kubectl apply

The root Application points at the apps/ folder. Once it exists, it creates every Application whose file is in that folder, so this is the one command that connects everything. checkout-dev already exists; root adopts it because its file is in apps/.

Apply the root Application. It is the only Application you ever create by hand:

kubectl apply -n argocd -f ~/work/argocd-platform/bootstrap/root.yaml

Give it a minute, then list the Applications. Expect three rows: root, plus its children argocd and checkout-dev:

argocd app list

Wait until Argo CD, now managing itself, is Synced and Healthy:

argocd app wait argocd --sync --health --timeout 300

4. Change Argo CD through Git

Add a banner to the UI: a visible change to Argo CD's own config, by pull request. After merge, Argo CD syncs itself and the banner appears (reload the UI).

cd ~/work/argocd-platform && git switch -q main && git pull -q

Add to ~/work/argocd-platform/argocd/values.yaml under cm:, after timeout.reconciliation: 60s

    # Module 04: a banner on every UI page.
    ui.bannercontent: "Course cluster: every change goes through Git"

Review your own change before you propose it. Expect only the banner lines, as additions:

git diff

Propose it as a pull request. Put it on its own branch and push it:

git switch -c ui-banner && git add -A && git commit -qm "Argo CD: UI banner"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

Ask Argo CD to look at Git, and wait until it has applied its own new settings:

argocd app get argocd --refresh >/dev/null
argocd app wait argocd --sync --health

The banner text is now in the ConfigMap. Expect Course cluster: every change goes through Git:

kubectl -n argocd get configmap argocd-cm -o jsonpath='{.data.ui\.bannercontent}'; echo

5. Look at the tree

In the UI open root: three Application nodes. Click through to argocd: Deployments, ConfigMaps, CRDs, all tracked. helm list still shows the old release record; it is harmless, but never run helm upgrade on it again.

The same tree in the terminal: everything root owns. Expect the Applications argocd and checkout-dev.

argocd app resources root

Helm still remembers its own install. Expect one release, argocd:

helm list -n argocd

Simplified for the course

In a team, the bootstrap (helm install plus the root app) is usually scripted or done by Terraform when the cluster is created: that is how Fleet GitOps does it. The rest is identical.

Break it on purpose: Tidy up the apps folder

A platform engineer tidies the platform repo and deletes apps/checkout-dev.yaml ("the root app has it now, right?"). The check passes, the pull request is merged.

You practise: what a file in apps/ really is, what deleting an Application does to what it deployed, and how to bring running resources back under Argo CD without downtime.

1. Break it

cd ~/work/argocd-platform && git switch -q main && git pull -q
git rm -q apps/checkout-dev.yaml

Propose it as a pull request. Put it on its own branch and push it:

git switch -c tidy-apps && git add -A && git commit -qm "Tidy up apps/"
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q

Give Argo CD a moment to notice:

argocd app get root --refresh >/dev/null; sleep 15

2. What you see

In Argo CD: checkout-dev is gone from the Applications page and from argocd app list.

In the cluster: kubectl -n checkout-dev get deploy,pods still shows checkout running. Nothing manages it any more. Try it: kubectl -n checkout-dev scale deployment checkout --replicas=4 stays at 4. The next real change would never arrive either.

3. Fix it yourself

Bring checkout under Argo CD again without deleting or restarting the running pods. Done when checkout-dev is Synced and Healthy and back at 2 replicas.

Hint 1

Who deleted the Application? Look at the root app's history. How do you undo a merged pull request?

Hint 2

When an Application is created for resources that already exist, Argo CD adopts them: it compares and fixes the difference instead of recreating them.

The fix
cd ~/work/argocd-platform && git switch -q main && git pull -q

Undo it the same way you ship it, with a pull request. Put the revert on a branch and push it:

git switch -c undo-tidy-apps && git revert --no-edit HEAD
git push -qu origin HEAD

Open the pull request. It prints a link:

gh pr create --fill

Open that link in your browser. Files changed shows exactly your diff. In the Checks tab, wait for validate to turn green (or run gh pr checks --watch).

Merge it in the portal (Squash and merge, then Delete branch), or from the terminal:

gh pr merge --squash --delete-branch

Either way, bring your copy up to date:

git switch -q main && git pull -q
argocd app get root --refresh >/dev/null; sleep 15
argocd app wait checkout-dev --sync --health

Expect 2/2: self-heal fixed the scale, and the pods were never deleted:

kubectl -n checkout-dev get deploy checkout
After you fixed it: the five questions
What failed?
Deleting an Application file removed the Application. It had no finalizer, so its workload kept running, unmanaged.
How was it detected?
Only by looking: the app disappears from the UI, but nothing fails and nothing alerts.
Who responds?
The platform team, who own the platform repo.
How is it corrected?
Revert the pull request. Root recreated the Application, and it adopted the running resources.
How is it prevented?
Review deletions in apps/ as uninstalls. Decide per Application, with the resources-finalizer.argocd.argoproj.io finalizer, whether deleting it also deletes its workload.

Quiz

  1. How do you add a new Application after this module?
    • Add its manifest to apps/ through a pull request
    • Run kubectl apply on its manifest once, and root will notice it on its next refresh
    • Create it by hand in the Argo CD UI, then export the YAML into Git for the record
  2. Why does root.yaml live in bootstrap/ and not in apps/?
    • It is only a naming style; nothing would change if root.yaml sat in apps/, as root can manage itself just as safely there
    • If root managed itself, a bad commit to its own file could delete it and everything below it
    • Argo CD refuses to load an Application whose manifest lives in the folder that same Application watches for changes
  3. How do you upgrade Argo CD from now on?
    • Run helm upgrade from a laptop with the new chart version, then commit the result to Git
    • Click upgrade in the UI settings page, which patches the running Deployment directly
    • A pull request changing targetRevision in apps/argocd.yaml
Answers
  1. Add its manifest to apps/ through a pull request The root app turns files in apps/ into Applications.
  2. If root managed itself, a bad commit to its own file could delete it and everything below it Keep the root out of its own source. It changes almost never and is applied by hand.
  3. A pull request changing targetRevision in apps/argocd.yaml Same as every other change. Module 06 does exactly that.

Field note

Teams worry that Argo CD managing itself is a loop that can lock them out. In practice it is the opposite: when a bad change breaks the UI, the controller keeps running and a revert in Git fixes it. Keep the bootstrap commands in the repo README for the day the controller itself is gone.

Report a problem with this module