PlatformEngr® Platform fundamentals course

Module 06 · Prove it · 20 min · You are: Platform team

Measuring value: outcomes, not output

In this module

  • You learn: DORA's five software delivery metrics, why adoption and developer experience are measured next to them, why a baseline comes before a target, and why counts of output (lines of code, pull requests, share of code by AI) are the wrong goal.
  • You do: Write the scorecard the platform team reports to leadership every month. Find the baselines you do not know yet: they are the first month's work. Then add the metric leadership asks for most, and move it to where it belongs.
  • You end with: metrics/scorecard.yaml passes the check: eight metrics, at least one each of delivery, stability, adoption, experience and cost, four baselines still unknown, and two metrics under avoid with the reason.

A platform that cannot show it helps loses its budget to one that can. The trap is to measure what is easy to count: more pull requests, more code, more of it written by AI. Those rise even when delivery gets worse. The DORA 2025 report describes AI's "primary role" as "an amplifier, magnifying an organization's existing strengths and weaknesses", so the question is not how much AI is used but whether outcomes improved. People are a poor guide too: in a 2025 study experienced developers "take 19% longer" with AI tools, while they "still believed AI had sped them up by 20%" (METR, July 2025). Measure, with a baseline.

Open this module in the interactive course to track progress.

Concepts

DORA's five metrics
Change lead time, deployment frequency and failed deployment recovery time (throughput); change fail rate and deployment rework rate (instability). "DORA has identified five software delivery performance metrics" (DORA metrics guide).
Baseline
The number today, before the platform changes anything. "unknown" is an honest baseline; measuring it is the first piece of work.
Outcome, not output
An outcome is what users and the business get (faster, safer changes). Output is what was produced (code, pull requests). Output can rise while outcomes fall.
Adoption and experience
Whether teams choose the path, and how it feels to use. The The SPACE of Developer Productivity, ACM Queue 2021 groups developer productivity into five dimensions; satisfaction and well-being is the first.
AI-assisted changes, compared
Label pull requests made with AI help and compare their change fail rate with the rest. Not "how much AI", but "is it as safe".

DORA's software delivery metrics

MetricGroupQuestion it answers
Change lead timeThroughputHow long from commit to running in production?
Deployment frequencyThroughputHow often do changes reach production?
Failed deployment recovery timeThroughputHow fast is a failed deployment recovered?
Change fail rateInstabilityWhat share of deployments need immediate fixing?
Deployment rework rateInstabilityWhat share of deployments were unplanned, to fix a problem?

From the DORA metrics guide, updated January 2026. Its metrics moved "from the original four keys to the current five-metric model".

Measure this, not that

MeasureInstead ofWhy
Change lead timeNumber of pull requests mergedSplitting work into more pull requests raises the count
Change fail rate of AI-assisted changesShare of code written by AIThe share rises with use, whether or not the changes help
Golden-path adoptionNumber of templates offeredTemplates nobody uses cost upkeep and help no one
Developer-reported frictionPortal page viewsA confusing portal gets more views

Files to know

Open them in your editor or on GitHub; the bullets say what to look at.

Steps

1. Write the scorecard

Eight metrics. Each has a source (where the number comes from), an owner, a baseline and a target.

cd ~/platformengr/platform-design

Create metrics/scorecard.yaml

# How the platform team knows the platform works: outcomes, not output.
# Owner: the platform team; reported to leadership every month. Read by the design check (rules M1 to M5).
# baseline: the number today, or "unknown" until it is measured.
metrics:
  - id: lead-time
    name: Change lead time (commit to production), median
    kind: delivery
    source: Deploy events joined to commits, from the delivery pipeline
    owner: platform
    baseline: 6 days
    target: 1 day
  - id: deploy-frequency
    name: Deployment frequency per service
    kind: delivery
    source: Deploy events
    owner: platform
    baseline: 2 per month
    target: daily
  - id: change-fail-rate
    name: Change fail rate, AI-assisted changes compared with the rest
    kind: stability
    source: Rollbacks and incidents linked to deploys; pull requests labelled ai-assisted
    owner: platform
    baseline: unknown
    target: AI-assisted no higher than the rest
  - id: recovery-time
    name: Failed deployment recovery time
    kind: stability
    source: Incident timeline
    owner: platform
    baseline: 4 hours
    target: under 1 hour
  - id: path-adoption
    name: Share of new services created on the golden path
    kind: adoption
    source: Service catalog, template field
    owner: platform
    baseline: 0%
    target: 80% of new services within two quarters
  - id: time-to-first-review
    name: Time to first review of a pull request
    kind: experience
    source: Code host pull request events
    owner: platform
    baseline: unknown
    target: under 4 working hours
  - id: developer-friction
    name: Developer-reported friction (quarterly survey, 1 to 5)
    kind: experience
    source: Quarterly developer survey, three questions
    owner: platform
    baseline: unknown
    target: 4 or better
  - id: cost-per-change
    name: Cloud and AI cost per merged change
    kind: cost
    source: Cost report per team divided by merged pull requests
    owner: platform
    baseline: unknown
    target: falls quarter on quarter

avoid:
  - metric: Number of pull requests merged
    why: Splitting work into more pull requests raises it; so does churn.

Run the check. Expect ok metrics/scorecard.yaml. Rule M4 checked that delivery, stability, adoption and experience are all measured, and, because agents have budgets in agents/policy.yaml, cost too.

npm run check

2. Find the first month's work

The baselines you do not know yet are what the platform team measures before it builds anything else.

Expect four lines: change fail rate, time to first review, developer friction and cost per change. For each, the source above it says where the number will come from.

grep -n -B5 "baseline: unknown" metrics/scorecard.yaml | grep "name:"

3. Commit

Save it.

git add -A && git commit -qm "Scorecard" && git push -q
gh run list --workflow design-check --limit 1

Simplified for the course

The numbers are examples. Yours come from your deploy pipeline, incident tool, code host and a short survey.

Eight metrics in one file. A real scorecard is a dashboard with history, and the file says which numbers it shows and why.

Deployment rework rate, DORA's fifth metric, is not on this scorecard. Add it once deploy events record whether a deployment was planned.

Break it on purpose: The metric leadership asks for

Leadership has read that other companies have half their code written by AI. They ask for that number on the scorecard, with a target of 50%, and a team lead adds it. Do this one, fix included: the next steps build on what the fix leaves in your repo.

You practise: turning a request for an output count into an outcome you can defend.

1. Break it

Add to metrics/scorecard.yaml under metrics:, after the cost-per-change metric

  - id: ai-code-share
    name: Share of code written by AI
    kind: adoption
    source: Coding assistant usage report
    owner: platform
    baseline: 20%
    target: 50%
npm run check

2. What you see

Expected: ok metrics/scorecard.yaml. You get: FAIL and a line with [M3] naming ai-code-share.

3. Fix it yourself

Answer leadership's question without making the share a goal. Done when the check passes, the share is listed where the team records what it will not be measured on, with the reason, and you can say which metric on the scorecard answers "is AI helping?"

Hint 1

What happens to the share if agents write a lot of code that is then rolled back?

Hint 2

Remove the metric and add it under avoid: with metric: and why:. Which existing metric compares AI-assisted changes with the rest?

The fix

change-fail-rate answers "is AI helping?": AI-assisted changes should fail no more often than the rest, while lead time falls.

Delete from metrics/scorecard.yaml

-   - id: ai-code-share
-     name: Share of code written by AI
-     kind: adoption
-     source: Coding assistant usage report
-     owner: platform
-     baseline: 20%
-     target: 50%

Add to metrics/scorecard.yaml under avoid:, as the first entry

  - metric: Share of code written by AI
    why: It rises when agents write more code, whether or not the change helps, and it says nothing about failures.
npm run check
git add -A && git commit -qm "Share of code by AI: avoided, with the reason" && git push -q
After you fixed it: the five questions
What failed?
The scorecard. It would have rewarded more AI-written code whatever it did to delivery.
How was it detected?
By rule M3 of the check.
Who responds?
The platform's product owner, with leadership.
How is it corrected?
Record it under avoid with the reason; point leadership at change fail rate of AI-assisted changes and lead time.
How is it prevented?
Agree the scorecard with leadership before the first report, including the list of what not to measure.

Quiz

  1. Which pair are both DORA throughput metrics?
    • Change fail rate and deployment rework rate
    • Change lead time and deployment frequency
    • Pull requests merged and lines of code changed
  2. Developers say AI tools make them about 20% faster. What does that tell the platform team?
    • Little on its own: a 2025 study found experienced developers felt faster while taking longer. Measure lead time with a baseline
    • That lead time has fallen by about 20%, so no measurement is needed
    • That the platform should buy more AI tools before measuring anything
  3. Why is "unknown" an acceptable baseline?
    • The check cannot read numbers, only words
    • Leadership prefers not to see bad numbers in the first report
    • It is honest, and it turns measuring the number into the first piece of work before any target is promised
Answers
  1. Change lead time and deployment frequency Throughput: lead time, deployment frequency, recovery time. Instability: change fail rate, rework rate.
  2. Little on its own: a 2025 study found experienced developers felt faster while taking longer. Measure lead time with a baseline METR: 19% longer, while believing 20% faster. Perception is not a measurement.
  3. It is honest, and it turns measuring the number into the first piece of work before any target is promised A made-up baseline makes every later improvement meaningless.

Field note

The first scorecard I wrote had twenty metrics and nobody read it. The one that worked had five, one of them was "unknown" for two months, and leadership trusted it more because of that.

Report a problem with this module