Cheat notes · Module 03 · Keep it healthy

Sizing: requests, limits, and a quota per team

A five-minute review for when you have done this module and want it back in your head. Open the full module for the steps and the lab, or the full text to print.

The idea

A request is what the scheduler reserves; a limit is what the node enforces. A ResourceQuota caps the sum for a namespace at admission, and a LimitRange fills in defaults. A pod refused by a quota never exists, so the reason is on the ReplicaSet, not on a pod.

Remember

Request: used to place the pod. Memory limit: over it the container is killed (OOMKilled, exit 137). CPU limit: throttles; many platforms leave it off.
QoS: BestEffort (nothing set) goes first under memory pressure, then Burstable, then Guaranteed.
A quota on requests.cpu refuses any pod that states no CPU request, unless a LimitRange gives a default.
Three places for "not running": no pod (ReplicaSet events, FailedCreate), Pending (pod events, FailedScheduling), restarts (Last State).
Size quotas for peak plus surge: a rolling update needs room for one extra pod.
checkout: requests 100m and 64Mi, limit 192Mi; quota 500m CPU, 8 pods, no NodePorts.

Pitfalls

Looking for a Pending pod when a quota said no: there is no pod. Read the ReplicaSet's FailedCreate events.
A quota set to exactly what is used: the next rollout cannot create its surge pod and times out, while the site stays up.
Expecting logs from an OOMKilled container: the kernel killed it, it printed nothing.

Commands

`kubectl -n NS describe resourcequota team-quota`	Used and hard, side by side
`kubectl -n NS get events --field-selector reason=FailedCreate`	Pods refused at admission
`kubectl -n NS describe pod POD \| grep -A4 'Last State'`	Why the last container died
`kubectl describe node NODE \| grep -A7 'Allocated resources'`	What the scheduler has given away on a node

tenants/checkout/quota.yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: checkout-dev
spec:
  hard:
    requests.cpu: 500m
    requests.memory: 512Mi
    limits.memory: 1Gi
    pods: "8"
    services.loadbalancers: "0"
    services.nodeports: "0"

Test yourself

Answer in your head first, then open the answer.

A Deployment asks for 6 replicas, the quota has room for 5. What do you see?

Five running pods and no sixth at all; the reason is in the ReplicaSet's events.

What happens to a container over its memory limit?

The kernel kills it and the kubelet restarts it: OOMKilled, exit code 137.

What is a request used for?

Placing the pod: the scheduler puts it only where the sum of requests fits.

Why did the rollout hang after the quota was right-sized?

A rolling update needs room for one extra pod, and the quota had none.

Where module 03 leaves you: Each checkout pod requests 100m CPU and 64Mi memory, with a 192Mi memory limit. The namespace has a ResourceQuota team-quota (500m CPU requested, 8 pods) and a LimitRange container-defaults. kubectl describe quota shows 200m of 500m used.