PlatformEngr® Terraform course

Cheat notes · Module 08 · Safe change

Debugging labs: five plans that go wrong

A five-minute review for when you have done this module and want it back in your head. Open the full module for the steps and the lab, or the full text to print.

The idea

Every Terraform failure is a disagreement between two of code, state and world. Name the two, read the one message that names the cause, and never fix a state problem by changing infrastructure.

Remember

  • Stale lock: Lock Info gives Who, Created, Operation. force-unlock <ID> only when that operation is dead.
  • Type rename (kubernetes_config_map to _v1): a new address. moved where the provider supports it, removed with destroy = false plus import where it does not.
  • Hand fix in prod: if the value in the cluster is right, put it in the code.
  • Error: Cycle: two resources refer to each other. Share a value both know through a local.
  • Invalid for_each argument: keys must be known at plan time. A map with keys you write, values that may be computed.
  • Never -lock=false, never -target to get past a design problem.

Pitfalls

  • Forcing a live lock: two writers, one corrupted state.
  • Following a deprecation warning by renaming the type: 2 to add, 2 to destroy.
  • for_each = toset([random_pet.x.id]): the keys depend on a value that exists only after apply.

Commands

terraform force-unlock LOCK_IDRemove a dead lock
pgrep -fl terraformIs the process in Who still running?
terraform plan -refresh-onlySeparate the hand change from yours

labs/upgrade/refactor.tf

moved {
  from = kubernetes_namespace.tools
  to   = kubernetes_namespace_v1.tools
}

removed {
  from = kubernetes_config_map.banner

  lifecycle {
    destroy = false
  }
}

import {
  to = kubernetes_config_map_v1.banner
  id = "platform-tools/banner"
}

Test yourself

Answer in your head first, then open the answer.

Lock Info shows an apply from 30 seconds ago. What do you do?

Wait or ask the person in Who; the lock is doing its job.

Why does renaming a resource type plan a destroy?

The type is part of the address.

How do you break a cycle between a namespace and a ServiceAccount?

Put the value both need, the account's name, in a local.

Which part of for_each must be known at plan time?

The keys.

Where module 08 leaves you: Every plan in envs/dev and envs/prod says No changes. labs/upgrade is on provider 3.5.0 with nothing recreated. Prod's checkout quota is 40 pods, in code. Every team namespace carries a platformengr.com/deployer annotation. Your repo has docs/runbook.md.