Module 01 · Run a workload · 45 min · You are: Platform and product

One app, three objects: Deployment, Service, Ingress

In this module

You learn: How a Deployment keeps a number of identical pods running, how a Service gives them one stable name by matching labels, and how an Ingress routes a hostname from outside the cluster to that Service.
You do: Install the ingress controller as the platform team. As team-checkout, add a Deployment, a Service and an Ingress for checkout, open it in a browser, release a new version, and delete a pod to watch it come back.
You end with: checkout runs in checkout-dev with 2 pods on image 6.17.0, and answers at http://checkout.localtest.me:8090. Your repo holds cluster/traefik.yaml and three files in apps/checkout/.

A container started by hand is gone when its node is, has an address that changes, and cannot be reached from outside. Three objects fix that, and they are joined by nothing but labels and names. That is flexible, and it is also the most common reason "everything is green and the site is down": each object is healthy on its own while the link between two of them is broken. You build the chain, then cut it.

Open this module in the interactive course to track progress.

Concepts

Pod: One or more containers that share a network address and live and die together. You almost never create one directly.
Deployment and ReplicaSet: A Deployment holds a pod template and a replica count. It creates a ReplicaSet per version of the template; the ReplicaSet creates the pods. A new version is a new ReplicaSet scaled up while the old one is scaled down.
Service: A stable name and virtual IP in front of every Ready pod whose labels match its selector. The list of matching pods is an EndpointSlice.
Ingress and its controller: An Ingress is a routing rule (host and path to a Service). It does nothing until an ingress controller, here Traefik, reads it. The platform team runs the controller.
Labels are the glue: Deployment to pods, Service to pods, later network rules and disruption budgets to pods: all by label selector. No object holds a reference that Kubernetes checks.

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

apps/checkout/deployment.yaml (reference, your repo): team-checkout's Deployment, in its finished form. Today you write the first 25 lines.
1. How many pods should run L11
2. Must match the template's labels, and cannot be changed later L12–14
3. A pinned image. podinfo, a small public demo app, stands in for the real checkout image L36
4. A named port: the Service and the probes refer to it by name L38–39
apps/checkout/service.yaml (reference, your repo): The stable address in front of the pods.
1. Which pods: every pod with this label L8–9
2. Listens on 80, forwards to the pods' port named http (9898) L12–13
apps/checkout/ingress.yaml (reference, your repo): The route from outside the cluster.
1. Which controller should act on it L8
2. Host and path to the Service L10–19
cluster/traefik.yaml (reference, your repo): The ingress controller: one file of ordinary Kubernetes objects, rendered from Traefik's Helm chart 42.4.0.
1. The class name an Ingress asks for L280–283
2. The port kind maps to 8090 on your laptop L138
3. Runs on the control plane, so draining a worker never moves the front door L258–259

Steps

1. Platform team: install the ingress controller

The ingress controller is the cluster's front door. The reference holds it as one file, rendered from Traefik's Helm chart at a pinned version, so you need no Helm.

cd ~/platformengr/k8s-platform && cp ~/platformengr/kubernetes-platform/cluster/traefik.yaml cluster/

Count what is in it. Expect seven kinds, one each: ClusterRole, ClusterRoleBinding, Deployment, IngressClass, Namespace, Service, ServiceAccount. An ingress controller is ordinary Kubernetes objects.

grep -E '^kind:' cluster/traefik.yaml | sort | uniq -c

Install it.

kubectl apply -f cluster/traefik.yaml

Wait until its pod is ready (it pulls an image first). Expect successfully rolled out:

kubectl -n traefik rollout status deployment/traefik --timeout=180s

Knock on the front door. Expect 404: Traefik answers and has no routes yet. Here 404 is the good result.

curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8090/

Save it in Git:

cd ~/platformengr/k8s-platform && git add -A && git commit -qm "Ingress controller: Traefik 42.4.0" && git push -q

2. team-checkout: the Deployment

Now you are team-checkout. Type or paste this file in your editor. Every object you write carries metadata.namespace: a file without it lands in whatever namespace your kube context points at, which is a classic way to deploy to the wrong place.

Create ~/platformengr/k8s-platform/apps/checkout/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout
  namespace: checkout-dev
  labels:
    app.kubernetes.io/name: checkout
spec:
  replicas: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: checkout
  template:
    metadata:
      labels:
        app.kubernetes.io/name: checkout
    spec:
      containers:
        - name: checkout
          image: ghcr.io/stefanprodan/podinfo:6.16.0
          ports:
            - name: http
              containerPort: 9898
          env:
            - name: PODINFO_UI_MESSAGE
              value: checkout (dev)

Apply it.

cd ~/platformengr/k8s-platform && kubectl apply -f apps/checkout/deployment.yaml

Wait for the pods. Expect successfully rolled out:

kubectl -n checkout-dev rollout status deployment/checkout --timeout=120s

Three kinds of object came from one file. Read the names: the ReplicaSet's name is the Deployment's plus a hash of the pod template; each pod's name is the ReplicaSet's plus five characters. The NODE column shows where each pod landed.

kubectl -n checkout-dev get deployment,replicaset,pod -o wide

3. Write the Service yourself

Pods come and go, and so do their addresses. A Service gives them one name, checkout, and finds the pods by label.

Create ~/platformengr/k8s-platform/apps/checkout/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: checkout
  namespace: checkout-dev
spec:
  selector:
    app.kubernetes.io/name: checkout
  ports:
    - name: http
      port: 80
      targetPort: http

cd ~/platformengr/k8s-platform && kubectl apply -f apps/checkout/service.yaml

The Service found two pods. ENDPOINTS lists their IP addresses with port 9898:

kubectl -n checkout-dev get endpointslices -l kubernetes.io/service-name=checkout

Look at the app through a port-forward. Run this in a new terminal and leave it running: it prints Forwarding from 127.0.0.1:9898 and waits. Then open http://localhost:9898 in your browser: the podinfo page with the banner checkout (dev). A port-forward is a debugging tunnel for you alone, not a way to expose an app. Stop it with Ctrl+C when you are done looking.

kubectl -n checkout-dev port-forward service/checkout 9898:80

4. Write the Ingress yourself

The Ingress tells Traefik: requests for checkout.localtest.me go to the Service checkout. localtest.me and every name under it point at your own machine, and port 8090 is the one kind maps to Traefik.

Create ~/platformengr/k8s-platform/apps/checkout/ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: checkout
  namespace: checkout-dev
spec:
  ingressClassName: traefik
  rules:
    - host: checkout.localtest.me
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: checkout
                port:
                  name: http

cd ~/platformengr/k8s-platform && kubectl apply -f apps/checkout/ingress.yaml

Ask checkout who it is. Run this three or four times: hostname changes, because the Service spreads requests over both pods. message is your banner.

curl -s http://checkout.localtest.me:8090/api/info | grep -E '"(hostname|version|message)"'

Open http://checkout.localtest.me:8090 in your browser too. Then save the three files in Git:

cd ~/platformengr/k8s-platform && git add -A && git commit -qm "checkout: Deployment, Service, Ingress" && git push -q

5. Release a new version

team-checkout ships podinfo 6.17.0. Change the image, look at the difference, then apply.

Change in ~/platformengr/k8s-platform/apps/checkout/deployment.yaml

-           image: ghcr.io/stefanprodan/podinfo:6.16.0
+           image: ghcr.io/stefanprodan/podinfo:6.17.0

Before you apply anything to a shared cluster, look at what will change. Expect one - line and one + line, the image (the generation counter may change too):

cd ~/platformengr/k8s-platform && kubectl diff -f apps/checkout/

cd ~/platformengr/k8s-platform && git add -A && git commit -qm "checkout: 6.17.0" && git push -q

Apply it.

kubectl apply -f apps/checkout/

kubectl -n checkout-dev rollout status deployment/checkout --timeout=120s

Two ReplicaSets now: the old one at 0 pods, the new one at 2. The old one is what a rollback would scale up again.

kubectl -n checkout-dev get replicaset

6. Delete a pod

Pods are disposable. Delete one and watch the ReplicaSet replace it.

kubectl -n checkout-dev delete "$(kubectl -n checkout-dev get pod -l app.kubernetes.io/name=checkout -o name | head -n 1)"

The ReplicaSet wanted 2 and saw 1. Expect a new pod with a new name and an AGE of a few seconds:

kubectl -n checkout-dev get pods

Simplified for the course

Ingress is stable but frozen: new routing features go into the Gateway API, where the platform team owns a Gateway and each team owns its HTTPRoute. The split of ownership is the same as here.
No TLS. In production the ingress controller ends TLS with certificates from cert-manager.
kubectl rollout undo exists and is not used here: it changes the cluster and leaves Git saying something else. Revert the commit and apply.
When you delete a file from Git, kubectl apply will not delete its object from the cluster. Argo CD calls the fix for that pruning.

Break it on purpose: A label tidy-up, half done

team-checkout decides to use the short label app: checkout everywhere. An engineer changes the Service first, because it is the smallest file, and plans to do the Deployment after lunch.

You practise: following the chain Ingress, Service, endpoints, pods, and finding the link that is broken when no object reports an error.

1. Break it

Change in ~/platformengr/k8s-platform/apps/checkout/service.yaml

-   selector:
-     app.kubernetes.io/name: checkout
+   selector:
+     app: checkout

cd ~/platformengr/k8s-platform && git add -A && git commit -qm "checkout: short labels" && git push -q

kubectl apply -f apps/checkout/service.yaml

2. What you see

Expected: curl -s -o /dev/null -w '%{http_code}\n' http://checkout.localtest.me:8090/ prints 200. You get: 503.

Now look for something red: kubectl -n checkout-dev get deployment,pod shows 2/2 and both pods Running, 1/1. No pod restarted. No event complains. Start at the front door and walk inwards, one object at a time.

3. Fix it yourself

Find the one place where the chain is cut, and say which command showed it. Then get 200 back with a commit.

Hint 1

The Ingress points at a Service; the Service points at pods. Which pods does the Service think it has? kubectl -n checkout-dev get endpointslices.

Hint 2

Compare kubectl -n checkout-dev describe service checkout | grep Selector with kubectl -n checkout-dev get pods --show-labels.

The fix

The Deployment's selector cannot be changed after creation, so the other way round (relabel the pods) means a new Deployment. The real tidy-up: add the new label to the pod template first, roll out, then switch the Service.

cd ~/platformengr/k8s-platform && git revert --no-edit HEAD && git push -q

kubectl apply -f apps/checkout/service.yaml

Expect 200:

curl -s -o /dev/null -w '%{http_code}\n' http://checkout.localtest.me:8090/

After you fixed it: the five questions

What failed?: The Service selected no pods, so it had no endpoints, so the ingress controller had nowhere to send requests.
How was it detected?: By a user, as HTTP 503. Kubernetes reported nothing: every object was valid and every pod healthy.
Who responds?: team-checkout: they own the Service and the labels.
How is it corrected?: Revert the selector.
How is it prevented?: A check from outside on the real URL, an alert on a Service with zero ready endpoints, and a linter in CI that flags a Service whose selector matches no pod template in the same folder.

Quiz

You change the image in a Deployment and apply it. What does Kubernetes create?
- New containers inside the existing pods, replaced one pod at a time in place
- A new ReplicaSet for the new pod template, scaled up while the old one is scaled down
- A second Deployment with the new image, which replaces the first one when it is ready
A Service has no endpoints although its pods are Running and Ready. What is the most likely cause?
- Its selector does not match the pods' labels
- The pods' image does not expose the port that the Service listens on
- The ingress controller has not picked up the new Service from the API yet
Who makes an Ingress object do something?
- The API server, which opens the host and port as soon as the object is created
- The kubelet on every node, which adds a routing rule for each Ingress it sees
- The ingress controller that matches its ingressClassName

Answers

A new ReplicaSet for the new pod template, scaled up while the old one is scaled down One ReplicaSet per template version; pods are never changed in place.
Its selector does not match the pods' labels Endpoints are computed from the selector and readiness only.
The ingress controller that matches its ingressClassName An Ingress is only data until a controller reads it.

Field note

When someone says the cluster is broken and every pod is green, I stop looking at pods. I look at what connects things: selectors, port names, the ingress class. It is almost always a name that is one character off.

Report a problem with this module