Cheat notes · Module 11 · Operate with teams

Debugging labs: seven ways Argo CD goes wrong

A five-minute review for when you have done this module and want it back in your head. Open the full module for the steps and the lab, or the full text to print.

The idea

Start at the Application and go one layer down at a time: status and conditions, the resource tree, the object's events, pod logs, then Argo CD's own logs. Render before you guess, and fix through Git.

Remember

Read the whole argocd app get before typing anything else. Sync, health, conditions and the last operation answer most questions.
Render or apply? Broken YAML fails at rendering (Unknown, ComparisonError, nothing applied). A wrong value such as memory: lots fails at applying (the sync fails).
The seven failures: a deny sync window holds a merge back; a bad image tag goes Synced, then Degraded; invalid manifests of both kinds; AppProject and RBAC refusals; a deleted deploy key; an Application stuck deleting; an autoscaler and Argo CD fighting over replicas.
Two owners, one field: pick one owner and make it permanent in Git. Here: declare the autoscaler in the values instead of creating it with kubectl.
Stuck deletion: find what the deletion waits for (another resource's finalizer) and fix that. Never strip Argo CD's own finalizer first.
Revert is the default fix. kubectl only restores service while the real fix goes through Git. Write one runbook entry per incident: symptom, where to look, cause, fix, prevention, owner.

Pitfalls

Removing resources-finalizer.argocd.argoproj.io to unstick a deletion leaves everything the app deployed behind, unmanaged.
Switching off self-heal to stop a fight over a field: the flapping stops, but Git no longer describes what runs.
Guessing before rendering. Nine times out of ten argocd app manifests or a local render shows the problem.

Commands

`argocd app get APP --refresh`	Status, conditions and the last operation
`argocd app manifests APP`	Exactly what Argo CD will apply
`argocd app diff APP`	What differs, field by field
`argocd proj windows list PROJECT`	Sync windows that can block syncs
`argocd repo list`	Repository connections and their status
`argocd app terminate-op APP`	Cancel a stuck sync

Test yourself

Answer in your head first, then open the answer.

An Application is Unknown with a ComparisonError. What is true?

Argo CD could not render or read the source. Nothing was applied and the running workload is untouched.

An app goes OutOfSync again right after every sync, always on the same field. Likely cause?

Another controller changes that field. Two owners for one field: pick one.

A change merged, but prod stays OutOfSync and never syncs. Where do you look?

Sync windows on the project: argocd proj windows list, or the Window line in argocd app get.

An Application is stuck deleting. What should you not do first?

Remove Argo CD's resources finalizer from it. Fix the stuck resource instead.

Where module 11 leaves you: All seven labs are cleaned up: the platform is healthy again and every Application is Synced and Healthy. Your platform repo has docs/runbook.md with one real entry.