Module 08 · Share the cluster · 60 min · You are: Platform and product

Capstone: a third team, on your own

A platform team is judged by what happens when the next team arrives. You have built two tenants by following steps. The third one has no steps: a list of requirements, the acceptance checks, and the files you already wrote. One requirement is new, and it has a trap that the network module already hinted at.

Open this module in the interactive course to track progress.

Concepts

The tenant kit

Namespace with Pod Security labels, quota, LimitRange, RBAC, network policies: the platform team's half. Deployment, Service, ConfigMap, budget: the team's half.

Order matters

Namespace first, then quota and defaults, then RBAC, then the app, then the network policies last, testing after each. A policy applied before the app works hides which one broke it.

An internal service

No Ingress, a ClusterIP Service, and an ingress policy that names exactly which pods may call it.

Both ends, again

Allowing checkout into payments is half the work. checkout's own egress policy must also let the call out.

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

• docs/onboarding.md (reference, your repo): The reference checklist. Write your own before you compare.
  1. What the platform team creates, in order, each with a check L6–22
  2. What the team brings L24–32
• tenants/payments/networkpolicy.yaml (reference, your repo): Do not open this before you are done.
  1. One list item with both selectors: pods named checkout AND in checkout-dev L16–33

Steps

1. Onboard team-payments (Challenge)

team-payments runs an internal service, payments. It takes card data, so nobody reaches it from outside the cluster, and inside only checkout's pods may call it. Build its tenant and its app the way you built the other two. Do not open tenants/payments or apps/payments in the reference until you are done. Requirements. (1) Namespace payments-dev, label team: payments, Pod Security restricted enforced. (2) Quota team-quota: room for 3 pods at 100m CPU plus one more during a rollout, and no more than 600m; at most 6 pods; no NodePort or LoadBalancer Services. A LimitRange with defaults. (3) Group team-payments may read the namespace; a ServiceAccount deployer may deploy into it; the workload runs as ServiceAccount payments with no token mounted. (4) payments: podinfo 6.17.0, 3 replicas, sized, three probes, spread over nodes, the restricted security context, banner payments (dev) from an immutable ConfigMap, a Service, a budget that lets a node be drained. No Ingress. (5) Network: payments-dev denies all incoming traffic except from pods labelled app.kubernetes.io/name: checkout in checkout-dev, on the pod port. Egress: DNS and its own namespace only. (6) And the part nobody mentions: checkout's own egress rules must let it call payments.

Done when:

• kubectl -n checkout-dev exec deployment/checkout -- wget -qO- -T 3 http://payments.payments-dev/api/info | grep message prints payments (dev)
• The same call from catalog-dev (kubectl -n catalog-dev exec deployment/catalog -- ...) ends in download timed out
• curl -s -o /dev/null -w '%{http_code}\n' http://payments.localtest.me:8090/ prints 404: no route from outside
• kubectl auth can-i list pods -n payments-dev --as=dev --as-group=team-payments says yes; get secrets in payments-dev says no; list pods -n checkout-dev says no
• kubectl auth can-i list pods -n payments-dev --as=alice --as-group=team-checkout says no
• payments reads its settings from an immutable ConfigMap payments-settings-v1, versioned in the name like checkout's
• kubectl -n payments-dev describe resourcequota team-quota shows requests.cpu 300m used and pods 3
• kubectl label --dry-run=server --overwrite namespace payments-dev pod-security.kubernetes.io/enforce=restricted prints no Warning:
• kubectl drain k8s-worker --ignore-daemonsets --delete-emptydir-data finishes while payments keeps 2 or more ready pods; then kubectl uncordon k8s-worker
• kubectl -n payments-dev exec deployment/payments -- ls /var/run/secrets/kubernetes.io/serviceaccount says No such file or directory

cd ~/platformengr/k8s-platform && diff -r ~/platformengr/kubernetes-platform/tenants/payments tenants/payments; diff -r ~/platformengr/kubernetes-platform/apps/payments apps/payments

cp -r ~/platformengr/kubernetes-platform/tenants/payments tenants/ && cp -r ~/platformengr/kubernetes-platform/apps/payments apps/

    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: payments-dev
          podSelector:
            matchLabels:
              app.kubernetes.io/name: payments
      ports:
        - protocol: TCP
          port: 9898

cd ~/platformengr/k8s-platform && git add -A && git commit -qm "Onboard team-payments; checkout may call payments" && git push -q

kubectl apply -f tenants/payments/namespace.yaml

kubectl apply -f tenants/payments/

kubectl apply -f apps/payments/

kubectl -n payments-dev rollout status deployment/payments --timeout=120s

kubectl apply -f tenants/checkout/networkpolicy.yaml

2. Write the onboarding checklist

The next team should not depend on your memory. Write what the platform team creates for a new team, what the team brings, in which order, and the command that proves each item with the answer you expect. Fill in the two lists.

Done when:

• A ## Platform team section and a ## Product team section
• Each item names the file it creates and a command that proves it
• At least one kubectl auth can-i line whose expected answer is no

Create ~/platformengr/k8s-platform/docs/onboarding.md

# Onboarding a new team

What the platform team creates, what the team brings, in which order, and the command that
proves each item (with the answer you expect).

## Platform team

1. ...

## Product team

1. ...

cd ~/platformengr/k8s-platform && git add -A && git commit -qm "docs: onboarding checklist" && git push -q

3. Clean up, and what comes next

Delete the cluster. Your repo is what you keep: a new cluster plus your repo, applied in order, gives you this platform back, except the one Secret listed in docs/secrets.md.

kind delete cluster --name k8s

Simplified for the course

Three teams were onboarded by copying a folder and changing a name. That works for three. For thirty, the folder is generated from one small file per team: see Fleet GitOps.

Each time you deleted a file you also had to delete the object by hand, and every apply was yours to remember. That is the gap Argo CD closes.

Quiz

1. payments accepts traffic from checkout's pods, yet the call from checkout times out. What is the first thing to check?
   • The payments Service's type, which must be NodePort for calls across namespaces
   • checkout's egress rules: both the caller's egress and the callee's ingress must allow the connection
   • The namespace's Pod Security level, which blocks traffic from pods in other namespaces
2. payments has 3 replicas. Which budget lets the platform team drain a node?
   • maxUnavailable: 1: one pod may be away, two keep serving
   • minAvailable: 3: all pods must stay, so the drain waits until it is safe
   • maxUnavailable: 0: nothing may be away, which is the safest setting for card data
3. In what order do you build a new tenant?
   • Network policies first, so that nothing is ever open, then the app, then the namespace
   • The app first, to show the team progress, then the quota and RBAC around it
   • Namespace, quota and defaults, RBAC, the app, then network policies, testing after each

Field note

If you finished this without the solution, you can onboard a team onto a shared cluster. If you needed it, that is normal: delete your payments folders, recreate the cluster from your repo next week, and do it again.

Report a problem with this module