Module 06 · Prove it · 20 min · You are: Platform team
Measuring value: outcomes, not output
In this module
- You learn: DORA's five software delivery metrics, why adoption and developer experience are measured next to them, why a baseline comes before a target, and why counts of output (lines of code, pull requests, share of code by AI) are the wrong goal.
- You do: Write the scorecard the platform team reports to leadership every month. Find the baselines you do not know yet: they are the first month's work. Then add the metric leadership asks for most, and move it to where it belongs.
- You end with:
metrics/scorecard.yamlpasses the check: eight metrics, at least one each of delivery, stability, adoption, experience and cost, four baselines stillunknown, and two metrics underavoidwith the reason.
A platform that cannot show it helps loses its budget to one that can. The trap is to measure what is easy to count: more pull requests, more code, more of it written by AI. Those rise even when delivery gets worse. The DORA 2025 report describes AI's "primary role" as "an amplifier, magnifying an organization's existing strengths and weaknesses", so the question is not how much AI is used but whether outcomes improved. People are a poor guide too: in a 2025 study experienced developers "take 19% longer" with AI tools, while they "still believed AI had sped them up by 20%" (METR, July 2025). Measure, with a baseline.
Open this module in the interactive course to track progress.
Concepts
- DORA's five metrics
- Change lead time, deployment frequency and failed deployment recovery time (throughput); change fail rate and deployment rework rate (instability). "DORA has identified five software delivery performance metrics" (DORA metrics guide).
- Baseline
- The number today, before the platform changes anything. "unknown" is an honest baseline; measuring it is the first piece of work.
- Outcome, not output
- An outcome is what users and the business get (faster, safer changes). Output is what was produced (code, pull requests). Output can rise while outcomes fall.
- Adoption and experience
- Whether teams choose the path, and how it feels to use. The The SPACE of Developer Productivity, ACM Queue 2021 groups developer productivity into five dimensions; satisfaction and well-being is the first.
- AI-assisted changes, compared
- Label pull requests made with AI help and compare their change fail rate with the rest. Not "how much AI", but "is it as safe".
DORA's software delivery metrics
| Metric | Group | Question it answers |
|---|---|---|
| Change lead time | Throughput | How long from commit to running in production? |
| Deployment frequency | Throughput | How often do changes reach production? |
| Failed deployment recovery time | Throughput | How fast is a failed deployment recovered? |
| Change fail rate | Instability | What share of deployments need immediate fixing? |
| Deployment rework rate | Instability | What share of deployments were unplanned, to fix a problem? |
From the DORA metrics guide, updated January 2026. Its metrics moved "from the original four keys to the current five-metric model".
Measure this, not that
| Measure | Instead of | Why |
|---|---|---|
| Change lead time | Number of pull requests merged | Splitting work into more pull requests raises the count |
| Change fail rate of AI-assisted changes | Share of code written by AI | The share rises with use, whether or not the changes help |
| Golden-path adoption | Number of templates offered | Templates nobody uses cost upkeep and help no one |
| Developer-reported friction | Portal page views | A confusing portal gets more views |
Files to know
Open them in your editor or on GitHub; the bullets say what to look at.
Steps
1. Write the scorecard
Eight metrics. Each has a source (where the number comes from), an owner, a baseline and a target.
cd ~/platformengr/platform-design
Create metrics/scorecard.yaml
# How the platform team knows the platform works: outcomes, not output.
# Owner: the platform team; reported to leadership every month. Read by the design check (rules M1 to M5).
# baseline: the number today, or "unknown" until it is measured.
metrics:
- id: lead-time
name: Change lead time (commit to production), median
kind: delivery
source: Deploy events joined to commits, from the delivery pipeline
owner: platform
baseline: 6 days
target: 1 day
- id: deploy-frequency
name: Deployment frequency per service
kind: delivery
source: Deploy events
owner: platform
baseline: 2 per month
target: daily
- id: change-fail-rate
name: Change fail rate, AI-assisted changes compared with the rest
kind: stability
source: Rollbacks and incidents linked to deploys; pull requests labelled ai-assisted
owner: platform
baseline: unknown
target: AI-assisted no higher than the rest
- id: recovery-time
name: Failed deployment recovery time
kind: stability
source: Incident timeline
owner: platform
baseline: 4 hours
target: under 1 hour
- id: path-adoption
name: Share of new services created on the golden path
kind: adoption
source: Service catalog, template field
owner: platform
baseline: 0%
target: 80% of new services within two quarters
- id: time-to-first-review
name: Time to first review of a pull request
kind: experience
source: Code host pull request events
owner: platform
baseline: unknown
target: under 4 working hours
- id: developer-friction
name: Developer-reported friction (quarterly survey, 1 to 5)
kind: experience
source: Quarterly developer survey, three questions
owner: platform
baseline: unknown
target: 4 or better
- id: cost-per-change
name: Cloud and AI cost per merged change
kind: cost
source: Cost report per team divided by merged pull requests
owner: platform
baseline: unknown
target: falls quarter on quarter
avoid:
- metric: Number of pull requests merged
why: Splitting work into more pull requests raises it; so does churn.
Run the check. Expect ok metrics/scorecard.yaml. Rule M4 checked that delivery, stability, adoption and experience are all measured, and, because agents have budgets in agents/policy.yaml, cost too.
npm run check
2. Find the first month's work
The baselines you do not know yet are what the platform team measures before it builds anything else.
Expect four lines: change fail rate, time to first review, developer friction and cost per change. For each, the source above it says where the number will come from.
grep -n -B5 "baseline: unknown" metrics/scorecard.yaml | grep "name:"
3. Commit
Save it.
git add -A && git commit -qm "Scorecard" && git push -q
gh run list --workflow design-check --limit 1
Simplified for the course
The numbers are examples. Yours come from your deploy pipeline, incident tool, code host and a short survey.
Eight metrics in one file. A real scorecard is a dashboard with history, and the file says which numbers it shows and why.
Deployment rework rate, DORA's fifth metric, is not on this scorecard. Add it once deploy events record whether a deployment was planned.
Break it on purpose: The metric leadership asks for
Leadership has read that other companies have half their code written by AI. They ask for that number on the scorecard, with a target of 50%, and a team lead adds it. Do this one, fix included: the next steps build on what the fix leaves in your repo.
You practise: turning a request for an output count into an outcome you can defend.
1. Break it
Add to metrics/scorecard.yaml under metrics:, after the cost-per-change metric
- id: ai-code-share
name: Share of code written by AI
kind: adoption
source: Coding assistant usage report
owner: platform
baseline: 20%
target: 50%
npm run check
2. What you see
Expected: ok metrics/scorecard.yaml. You get: FAIL and a line with [M3] naming ai-code-share.
3. Fix it yourself
Answer leadership's question without making the share a goal. Done when the check passes, the share is listed where the team records what it will not be measured on, with the reason, and you can say which metric on the scorecard answers "is AI helping?"
Hint 1
What happens to the share if agents write a lot of code that is then rolled back?
Hint 2
Remove the metric and add it under avoid: with metric: and why:. Which existing metric compares AI-assisted changes with the rest?
The fix
change-fail-rate answers "is AI helping?": AI-assisted changes should fail no more often than the rest, while lead time falls.
Delete from metrics/scorecard.yaml
- - id: ai-code-share
- name: Share of code written by AI
- kind: adoption
- source: Coding assistant usage report
- owner: platform
- baseline: 20%
- target: 50%
Add to metrics/scorecard.yaml under avoid:, as the first entry
- metric: Share of code written by AI
why: It rises when agents write more code, whether or not the change helps, and it says nothing about failures.
npm run check
git add -A && git commit -qm "Share of code by AI: avoided, with the reason" && git push -q
After you fixed it: the five questions
- What failed?
- The scorecard. It would have rewarded more AI-written code whatever it did to delivery.
- How was it detected?
- By rule
M3of the check. - Who responds?
- The platform's product owner, with leadership.
- How is it corrected?
- Record it under
avoidwith the reason; point leadership at change fail rate of AI-assisted changes and lead time. - How is it prevented?
- Agree the scorecard with leadership before the first report, including the list of what not to measure.
Quiz
- Which pair are both DORA throughput metrics?
- Change fail rate and deployment rework rate
- Change lead time and deployment frequency
- Pull requests merged and lines of code changed
- Developers say AI tools make them about 20% faster. What does that tell the platform team?
- Little on its own: a 2025 study found experienced developers felt faster while taking longer. Measure lead time with a baseline
- That lead time has fallen by about 20%, so no measurement is needed
- That the platform should buy more AI tools before measuring anything
- Why is "unknown" an acceptable baseline?
- The check cannot read numbers, only words
- Leadership prefers not to see bad numbers in the first report
- It is honest, and it turns measuring the number into the first piece of work before any target is promised
Answers
- Change lead time and deployment frequency Throughput: lead time, deployment frequency, recovery time. Instability: change fail rate, rework rate.
- Little on its own: a 2025 study found experienced developers felt faster while taking longer. Measure lead time with a baseline METR: 19% longer, while believing 20% faster. Perception is not a measurement.
- It is honest, and it turns measuring the number into the first piece of work before any target is promised A made-up baseline makes every later improvement meaningless.
Field note
The first scorecard I wrote had twenty metrics and nobody read it. The one that worked had five, one of them was "unknown" for two months, and leadership trusted it more because of that.