PlatformEngr® Fleet GitOps course

Cheat notes · Module 04 · Platform foundation

Platform add-ons on every cluster

A five-minute review for when you have done this module and want it back in your head. Open the full module for the steps and the lab, or the full text to print.

The idea

One matrix ApplicationSet, every cluster times every add-on, puts the same pinned add-ons on all clusters. Consistency becomes a property of the design, and a new cluster gets everything for free.

Remember

  • Matrix generator: combines two generators, clusters from the inventory times a list of add-ons. 2 clusters × 7 add-ons = 14 Applications from one file.
  • Pinned versions: every chart has an exact targetRevision. Renovate opens pull requests to bump them, through the same validate and plan.
  • Order is eventual: sync waves express intent (CRDs before their users), but generated Applications are independent. Something that needs a CRD fails and retries until it exists.
  • Add a cluster, get everything: a new inventory file produces all its add-on Applications; the plan comment lists them before you merge.
  • The add-ons: ingress, cert-manager, external-secrets, metrics, Kyverno, Argo Rollouts and more, labelled platform-addon.

Pitfalls

  • Adding the same add-on twice with new values: duplicate Application names (dev-kyverno). CI catches it; without the check the ApplicationSet controller would refuse and report an error. One entry per add-on, values in that entry.
  • Floating chart versions: a chart that moves to a new major on its own is a classic Friday outage. Pin and let a bot propose bumps.
  • On kind, Docker needs about 8 GB of memory for every add-on.

Commands

python3 scripts/fleet_check.py | head -40Every cluster with the add-ons it will get
kubectl --context kind-platform -n argocd get applications -l platform-addon -wWatch the add-on Applications
kubectl --context kind-dev get pods -AThe add-ons running on a spoke

Test yourself

Answer in your head first, then open the answer.

How many Applications does a matrix of 3 clusters and 7 add-ons generate?

21: every combination.

How do chart versions get updated?

Renovate opens a pull request that bumps the pinned version.

An Application needs a CRD another Application installs, and syncs first. What happens?

It fails and retries until the CRD exists. Ordering between generated Applications is eventual.

Where module 04 leaves you: Every cluster runs the same pinned platform add-ons (ingress, certificates, secrets, metrics, policy engine, rollouts): 14 add-on Applications from one file, kept up to date by version-bump pull requests.