Module 09 · Operate with teams · 50 min · You are: Platform team

AppProjects and RBAC: team boundaries

So far everything ran in the default project, which allows any repo, any namespace, any kind, and you were admin. With a second team on the cluster that is not a platform, it is a shared root account. AppProjects decide what Argo CD may deploy for a team; RBAC decides what a person may do in Argo CD.

Open this module in the interactive course to track progress and switch tracks.

GitOps loop practised: Committed → Reviewed → Detected → Rendered → Compared → Synced → Healthy → Observed

Concepts

AppProject

Allowed sourceRepos, allowed destinations (cluster and namespace), cluster-scoped kinds allowed, namespaced kinds denied. Argo CD refuses to sync anything outside it.

Argo CD RBAC

policy.csv: roles with permissions on applications, projects, logs, and which accounts or groups have them. policy.default: "" means no access unless granted.

Platform-owned namespaces

The team deploys into namespaces it cannot create, label or delete. CreateNamespace goes away for team apps.

Project roles and sync windows

Project roles issue tokens for automation scoped to one project. Sync windows allow or deny syncs on a schedule (module 11 uses one).

Two fences, two questions

Product teams in a GitOps setup usually get read-only kubectl or none: their write path is Git.

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

• platform/projects/checkout.yaml (reference): team-checkout's boundary.
  1. Only its two repos L9–11
  2. Only its two namespaces L12–16
  3. Nothing cluster-wide L18
  4. Nothing the platform owns inside them L20–30
• platform/projects/platform.yaml (reference): The platform team's own project: wide, but listed.
  1. Every repo the platform deploys from, explicitly L9–13
• platform/argocd/values.yaml (reference): The team account and its role.
  1. A local account for a team member L16
  2. See, sync, run actions and read logs in project checkout only L41–47
• platform/namespaces/checkout.yaml (reference): Namespaces, owned by the platform.
  1. Labels the platform controls L6–9

Steps

1. Projects and namespaces, platform-owned

Two new platform-owned Applications: one creates the AppProjects (the fences around each team), the other creates the namespaces. The copied files are marked to sync first, so the fences and namespaces exist before anything that uses them.

Copy the project and namespace files, and the two Applications that deploy them:

cd ~/work/argocd-platform && git switch -q main && git pull -q
cp -r ~/work/argocd-gitops/platform/projects ~/work/argocd-gitops/platform/namespaces .
cp ~/work/argocd-gitops/platform/apps/projects.yaml ~/work/argocd-gitops/platform/apps/namespaces.yaml apps/

Put your GitHub user in place of the placeholder:

perl -pi -e 's/YOUR_GITHUB_USER/YOUR_GITHUB_USER/g' projects/*.yaml apps/projects.yaml apps/namespaces.yaml

Change in ~/work/argocd-platform/apps/projects.yaml, ~/work/argocd-platform/apps/namespaces.yaml
For now: the next step moves everything into its project at once.

-   project: platform
+   project: default

git add -A && git commit -qm "AppProjects and platform-owned namespaces" && git push -q

argocd app get root --refresh >/dev/null
argocd app wait projects namespaces --sync --health

List the projects. Expect three: default, platform and checkout:

argocd proj list

2. Move every Application into its project

Platform apps move into the platform project, checkout apps into checkout, and checkout stops creating its own namespaces. The root file lives in bootstrap/, outside the folder root watches, so this one change to it must be applied by hand.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Change in ~/work/argocd-platform/bootstrap/root.yaml, apps/argocd.yaml, apps/traefik.yaml, apps/sealed-secrets.yaml, apps/repo-credentials.yaml, apps/projects.yaml, apps/namespaces.yaml
In all seven platform files.

-   project: default
+   project: platform

Change in ~/work/argocd-platform/apps/checkout-dev.yaml, ~/work/argocd-platform/apps/checkout-prod.yaml

-   project: default
+   project: checkout

Delete from ~/work/argocd-platform/apps/checkout-dev.yaml, ~/work/argocd-platform/apps/checkout-prod.yaml
The last two lines of both files: the platform owns the namespaces now.

-     syncOptions:
-       - CreateNamespace=true

git add -A && git commit -qm "Every Application in its project" && git push -q

Apply the root file by hand. Argo CD updates every Application from apps/ by itself, but not root, because its file is outside that folder. This command updates it.

kubectl apply -n argocd -f bootstrap/root.yaml

argocd app get root --refresh >/dev/null; sleep 20

List every Application with its project. Expect checkout-dev and checkout-prod in checkout, and all the others in platform:

argocd app list -o wide | awk '{print $1, $2, $5, $6}' | column -t

3. An account for team-checkout

Declare it and its role in Git; set the password as alice.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Add to ~/work/argocd-platform/argocd/values.yaml under cm:, after accounts.alice: login

    accounts.checkout-dev: login     # team-checkout (module 09)

Add to ~/work/argocd-platform/argocd/values.yaml at the end of policy.csv, after g, alice, role:admin

      # team-checkout: its own project only (module 09).
      p, role:checkout-team, applications, get, checkout/*, allow
      p, role:checkout-team, applications, sync, checkout/*, allow
      p, role:checkout-team, applications, action/*, checkout/*, allow
      p, role:checkout-team, logs, get, checkout/*, allow
      p, role:checkout-team, projects, get, checkout, allow
      g, checkout-dev, role:checkout-team

Expect only added lines: the account, and the role with its rules:

git diff

git add -A && git commit -qm "Argo CD: team-checkout account and role" && git push -q

argocd app get argocd --refresh >/dev/null
argocd app wait argocd --sync --health

Set the password. You are logged in as alice, so alice is allowed to do this for others (the first value is alice's own password):

argocd account update-password --account checkout-dev --current-password 'choose-a-long-one' --new-password 'another-long-one'

4. Be team-checkout for a minute

The test of a fence is trying to cross it. Log in as the team member and try things they should and should not be able to do.

argocd login argocd.localtest.me:8443 --grpc-web --insecure --username checkout-dev --password 'another-long-one'

Allowed: see your apps. Expect only checkout-dev and checkout-prod. The platform apps are invisible to this account.

argocd app list

Allowed: sync your app.

argocd app sync checkout-dev

Allowed: read your logs. Expect three log lines:

argocd app logs checkout-dev --tail 3

Not allowed: delete prod. Expect permission denied:

argocd app delete checkout-prod --yes

Not allowed: anything in the platform project. Expect no:

argocd account can-i sync applications 'platform/*'

Log back in as alice, the administrator:

argocd login argocd.localtest.me:8443 --grpc-web --insecure --username alice --password 'choose-a-long-one'

Simplified for the course

The default project still exists and is wide open. In production, restrict it (sourceRepos: []) so nothing can use it by accident, and map SSO groups instead of local accounts in policy.csv.

Break it on purpose: team-checkout asks for cluster-wide access

A team-checkout developer wants to read everything in the cluster while debugging, and adds a ClusterRole to the team's repo.

You practise: reading a project refusal, and turning an access request into the narrowest grant, made by the team that owns access.

1. Break it

cd ~/work/checkout-config && git switch -q main && git pull -q

Create ~/work/checkout-config/secrets/dev/clusterrole.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: checkout-read-everything
rules:
  - apiGroups: ["*"]
    resources: ["*"]
    verbs: ["get", "list"]

git add -A && git commit -qm "checkout: cluster read access" && git push -q

argocd app get checkout-dev --refresh >/dev/null; sleep 10

2. What you see

Expected (by the developer): the ClusterRole is created. You get: checkout-dev is OutOfSync and its sync failed. argocd app get checkout-dev says the ClusterRole is not permitted in project checkout. Nothing was applied; the running app is untouched.

3. Fix it yourself

Remove the refused object so checkout-dev syncs again. Then, as the platform team, give team-checkout (the Kubernetes group team-checkout) read access to its own two namespaces and nothing else. Prove it with kubectl auth can-i.

cd ~/work/checkout-config
git rm -q secrets/dev/clusterrole.yaml

git add -A && git commit -qm "checkout: remove the ClusterRole" && git push -q

cd ~/work/argocd-platform && git switch -q main && git pull -q

# team-checkout may read everything in its own namespaces. Granted by the platform team.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-checkout-view
  namespace: checkout-dev
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: team-checkout
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: team-checkout-view
  namespace: checkout-prod
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: view
subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: team-checkout

git add -A && git commit -qm "team-checkout: read access to its namespaces" && git push -q

argocd app get namespaces --refresh >/dev/null; sleep 15

kubectl auth can-i list pods -n checkout-dev --as jane --as-group team-checkout

kubectl auth can-i list pods -n kube-system --as jane --as-group team-checkout

Quiz

1. team-checkout wants a NetworkPolicy in its namespace. With this project, what happens?
   • Argo CD refuses: NetworkPolicy is on the project's namespace blacklist
   • It is applied normally, as AppProjects only restrict which repositories an Application may use
   • kubectl applies it, since the project only controls what the UI shows, not what is deployed
2. checkout-dev runs argocd app delete checkout-prod. Which fence stops it?
   • The AppProject, since it lists which Applications a team may delete from its destinations
   • Argo CD RBAC: its role has get, sync, actions and logs only
   • Kubernetes RBAC on the argocd namespace, which blocks deleting Application objects for that user
3. Why do namespaces belong to the platform?
   • Argo CD itself cannot create namespaces, so a person with cluster-admin access has to create each one by hand before any sync can work
   • They are a performance concern: letting many teams create namespaces slows the API server, so one owner keeps the total number of them low
   • Labels, quotas and policies on the namespace are the platform's guarantees; a team must not be able to remove them

Field note

Teams always ask for "just admin in dev". Give them a project per environment if dev needs to be looser, but keep the fence. The habits a team forms in dev are the ones it brings to prod.

Report a problem with this module