PlatformEngr® Progressive delivery course

Module 07 · Traffic · 30 min · You are: Platform and product

Gateway API: the lighter option

In this module

  • You learn: What the Gateway API is, how a plugin lets Argo Rollouts drive an HTTPRoute, and the trade-offs between pod count, a mesh and a gateway.
  • You do: Install the Gateway API CRDs, turn on Traefik's Gateway API provider, add the plugin to Argo Rollouts, switch dev from blue-green to a canary through an HTTPRoute, and ship a release at 20%.
  • You end with: Dev releases through HTTPRoute/checkout on Traefik's traefik-gateway: 20% and 50% steps with two pods, judged by success-rate. Dev is served at http://checkout-dev.localtest.me:8088 and runs 6.12.0.

A mesh is a big decision to make for a canary. Most clusters already have an ingress controller, and most of them now speak the Gateway API, a standard routing API with weights built in. Splitting at the edge is enough when the traffic you want to protect comes from outside: users through the gateway.

Open this module in the interactive course to track progress and switch tracks.

Release path practised: Committed → Deployed → Exposed → Measured → Decided → Promoted → Rolled back

Concepts

Gateway API
Kubernetes' standard routing API: a GatewayClass (the implementation), a Gateway (a listener, the platform's) and HTTPRoutes (a team's routes, with weighted backends).
Plugin
Argo Rollouts supports some routers itself (Istio) and others through plugins. The controller downloads a pinned plugin binary when it starts.
allowedRoutes
The Gateway says which namespaces may attach routes. Here: namespaces labelled team: checkout.
Edge split
Only requests through the gateway are split. A pod calling checkout.checkout-dev directly reaches every pod by Service: no canary share there.

Three ways to split a canary

Pod countIstioGateway API
Smallest step1 pod of Nany percentageany percentage
Which traffic is splitall, roughlyevery call in the meshonly through the gateway
Header route for testersnoyesyes (plugin)
Needsnothingsidecars in the namespacea Gateway API implementation and the plugin
ExtrasnonemTLS, per-call metricsthe same API across Traefik, Istio, Cilium, cloud gateways

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

Steps

1. The Gateway API CRDs and Traefik's provider

The CRDs first (their own release), then Traefik, which creates the GatewayClass traefik and the Gateway traefik-gateway.

cd ~/work/argocd-platform && git switch -q main && git pull -q

Add to ~/work/argocd-platform/projects/platform.yaml under sourceRepos:, after the istio line

    - https://github.com/kubernetes-sigs/gateway-api.git    # module 07
cp ~/work/progressive-delivery/platform/apps/gateway-api.yaml apps/

Add to ~/work/argocd-platform/apps/traefik.yaml under valuesObject:, after isDefaultClass: true, at the same indentation as ingressClass:

        # Module 07: Traefik also implements the Gateway API. The chart creates
        # the GatewayClass "traefik" and a Gateway "traefik-gateway" on the
        # web entrypoint; checkout's namespaces may attach routes to it.
        providers:
          kubernetesGateway:
            enabled: true
        gateway:
          listeners:
            web:
              namespacePolicy:
                from: Selector
                selector:
                  matchLabels:
                    team: checkout
scripts/validate.sh
git add -A && git commit -qm "Gateway API v1.5.1; Traefik as its implementation" && git push -q
argocd app get root --refresh >/dev/null
argocd app wait gateway-api --sync --health --timeout 300
argocd app wait traefik --sync --health --timeout 300

Expect the class traefik with ACCEPTED True and the Gateway traefik-gateway with PROGRAMMED True:

kubectl get gatewayclass,gateway -A

2. The plugin for Argo Rollouts

The controller downloads the plugin when it starts. The chart restarts it when its config changes.

Add to ~/work/argocd-platform/argo-rollouts/values.yaml under controller:, after replicas: 1

  # Module 07: traffic router plugins. The controller downloads the plugin
  # when it starts; the version is pinned in the URL.
  trafficRouterPlugins:
    - name: argoproj-labs/gatewayAPI
      location: https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.17.0/gatewayapi-plugin-linux-amd64
git add -A && git commit -qm "Argo Rollouts: Gateway API plugin v0.17.0" && git push -q
argocd app get argo-rollouts --refresh >/dev/null
argocd app wait argo-rollouts --sync --health --timeout 300

Expect a log line that the plugin was downloaded and loaded:

kubectl -n argo-rollouts logs deploy/argo-rollouts | grep -i plugin | tail -3

3. Move dev to a canary on the gateway

Dev leaves blue-green: the strategy, the steps and the host change in values, and the Application ignores the route's weights.

Platform: ignore the HTTPRoute weights in the dev Application, as for prod.

Add to ~/work/argocd-platform/apps/checkout-dev.yaml above syncPolicy:, at the same indentation

  # Module 07: the Gateway API plugin moves the weights during a release.
  ignoreDifferences:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      jqPathExpressions:
        - .spec.rules[].backendRefs[].weight

Add to ~/work/argocd-platform/apps/checkout-dev.yaml at the end of the file, under syncPolicy: (after selfHeal: true)

    syncOptions:
      - RespectIgnoreDifferences=true
git add -A && git commit -qm "checkout-dev: ignore HTTPRoute weights" && git push -q
argocd app get root --refresh >/dev/null

team-checkout: in dev's values, replace everything from strategy: blueGreen to the end of the file with the block below (the Ingress hosts go: the route serves the host now).

cd ~/work/checkout-config && git switch -q main && git pull -q

Change in ~/work/checkout-config/helm/dev.yaml

- strategy: blueGreen
- blueGreen:
-   autoPromotionEnabled: true     # analysis decides now, not a person
-   scaleDownDelaySeconds: 30
-   prePromotionAnalysis:
-     templates: [success-rate]    # the platform's ClusterAnalysisTemplate
- ingress:
-   host: checkout-dev.localtest.me            # the live version
-   previewHost: preview-checkout-dev.localtest.me   # the next one, before promotion
+ # Module 07: a canary through the Gateway API on Traefik, no mesh.
+ strategy: canary
+ canary:
+   trafficRouting: gatewayapi
+   steps:
+     - setWeight: 20            # 20% of requests, even with two pods
+     - pause: { duration: 1m }
+     - setWeight: 50
+     - pause: { duration: 1m }
+   analysis:
+     templates: [success-rate]
+     startingStep: 1
+ gatewayApi:
+   host: checkout-dev.localtest.me   # http://checkout-dev.localtest.me:8088
git add -A && git commit -qm "checkout dev: canary through the Gateway API" && git push -q
argocd app get checkout-dev --refresh >/dev/null
argocd app wait checkout-dev --sync --health --timeout 300

Expect the route Accepted by its Gateway, and an answer on the web port:

kubectl -n checkout-dev get httproute checkout -o jsonpath='{.status.parents[0].conditions[?(@.type=="Accepted")].status}'; echo
curl -s http://checkout-dev.localtest.me:8088/ | jq -r .version

4. Ship 6.12.0 to dev at 20%

Two pods, and still a 20% step: the gateway splits requests.

In a second terminal, watch:

kubectl argo rollouts get rollout checkout -n checkout-dev --watch
cd ~/work/checkout-config && git switch -q main && git pull -q

Change in ~/work/checkout-config/helm/dev.yaml

- tag: "6.11.1"
+ tag: "6.12.0"
git add -A && git commit -qm "checkout dev: 6.12.0" && git push -q
argocd app get checkout-dev --refresh >/dev/null

During the first pause, expect the weights 80 20:

kubectl -n checkout-dev get httproute checkout -o jsonpath='{.spec.rules[0].backendRefs[*].weight}'; echo

Twenty requests through the gateway: about four answer 6.12.0:

for i in $(seq 20); do curl -s http://checkout-dev.localtest.me:8088/ | jq -r .version; done | sort | uniq -c

Two minutes later the watch ends ✔ Healthy, and the weights are 100 0 again, now pointing at the new stable pods.

Simplified for the course

The plugin is downloaded from GitHub when the controller starts. In production, mirror the binary inside your network (or bake it into an image) and point location there, with a checksum.

Break it on purpose: A Gateway that is not there

team-checkout copies values from another cluster where the Gateway is called public. They set gatewayApi.gateway: public in dev.

You practise: reading a route's status, the one place that says why a Gateway API route serves nothing.

1. Break it

cd ~/work/checkout-config && git switch -q main && git pull -q

Add to ~/work/checkout-config/helm/dev.yaml under gatewayApi:, after host:

  gateway: public
git add -A && git commit -qm "checkout dev: gateway from the shared values" && git push -q
argocd app get checkout-dev --refresh >/dev/null

2. What you see

Expected: dev unchanged at http://checkout-dev.localtest.me:8088. You get: 404 page not found from Traefik. Argo CD says checkout-dev is Synced and Healthy: the HTTPRoute was applied without error.

Look at the HTTPRoute's status (kubectl -n checkout-dev get httproute checkout -o yaml).

3. Fix it yourself

Find why the route serves nothing and make dev answer again through Git. Done when the curl answers with checkout's JSON and the route's Accepted condition is True.

Hint 1

Under status.parents, the condition Accepted is False or missing. Which parent does the route name?

Hint 2

There is no Gateway public in traefik. Remove the gateway: line so the chart default (traefik-gateway) applies.

The fix
cd ~/work/checkout-config && git switch -q main && git pull -q
git revert --no-edit HEAD && git push -q
argocd app get checkout-dev --refresh >/dev/null
curl -s http://checkout-dev.localtest.me:8088/ | jq -r .version
After you fixed it: the five questions
What failed?
A route attached to a Gateway that does not exist.
How was it detected?
Users: 404 from the gateway. The route's status: not accepted. Argo CD has no health check for routes, so it stayed green.
Who responds?
team-checkout, who own the values.
How is it corrected?
The right Gateway in values (here: the default).
How is it prevented?
A custom health check in Argo CD for HTTPRoute that reads status.parents[].conditions (Degraded when not Accepted), so a bad route turns the Application red.

Quiz

  1. A pod in checkout-dev calls http://checkout during a Gateway API canary. Which version does it get?
    • The canary share, like users
    • Any pod behind the Service: the split happens only at the gateway
    • Always the stable version
  2. Where does the platform say which teams may attach routes to its Gateway?
    • In each team's HTTPRoute
    • In the Gateway's allowedRoutes, here namespaces labelled team: checkout
    • In the AppProject
  3. When is the Gateway API the better choice over Istio for a canary?
    • When the traffic to protect comes from outside, and there is no mesh yet
    • When you need mTLS between services
    • When calls between services must be split too
Answers
  1. Any pod behind the Service: the split happens only at the gateway Edge splitting only applies to requests through the Gateway.
  2. In the Gateway's allowedRoutes, here namespaces labelled team: checkout The Gateway is the platform's; it decides who may use it.
  3. When the traffic to protect comes from outside, and there is no mesh yet A gateway split protects users at the edge without sidecars.

Field note

I recommend the Gateway API to teams without a mesh, because the same HTTPRoute works on the next gateway too: when Traefik is swapped for a cloud load balancer, the Rollout does not change.

Report a problem with this module