Reporting & the Release Decision
Friday, 4 p.m. Petra turns to you: "So — can we ship?" Everything in this book funnels into that question, and here is the answer that defines the profession: never "yes," never "no." The professional answer is an honest picture — what was tested, what was found, what risks remain — delivered so the people who own the decision can actually make it. This topic is that picture, and the role clarity behind it.
The right model is the weather briefing before a flight. The meteorologist doesn't decide whether the plane takes off — the pilot does. The meteorologist's job is to make sure nobody decides while blind. And one more thing about that room: a pilot who shames the forecaster for reporting storms gets exactly one honest briefing, ever. Remember both halves; they're the whole politics of this page.
The One-Page Test Report
The release-time test report is written for Petra and Nadia — decision-makers, not testers — and it answers four questions on one page. What was covered: the risk-table areas, with their depths — and, the courage line, what was not: "mobile regression was cut Monday per the scope call." What was found: pass/fail/blocked counts with meaning attached — "3 blocked on the Paylane sandbox" is information; a bare "3 blocked" is trivia. What's still open: bugs by severity, each with a one-line consequence. What risks remain: named, specific, owned: "promo revalidation (FW-372) is open — modified bookings can carry invalid discounts until the hotfix." No poetry, no padding; the reader has a decision to make and four minutes.
Metrics, Honestly
Numbers help decisions when they inform and hurt when they perform. Informative: open bugs by severity (what's actually wrong), coverage of the risk-ranked areas (was the plan executed), re-open rate (are fixes actually fixing). Vanity: total test cases executed (thirty same-partition repeats count as thirty), bugs-found-per-tester (watch colleagues start filing typos as three tickets each). The trap has a name — Goodhart's law: when a measure becomes a target, it stops measuring. Make "bugs found" a KPI and you'll get more bugs found — smaller, split, gamed. At beginner altitude the rule is simple: report numbers that describe the product's state, never numbers that describe testers' output. The first kind informs Friday; the second kind corrupts Monday.
Exit Criteria, Revisited
Now Chapter 5's promise comes due. The exit criteria were agreed at planning, in calm: no open critical bugs; risk-tier-1 areas covered to plan; regression green. So Friday's meeting isn't a negotiation about feelings — it's a check against written criteria. Two are met; one has an asterisk (FW-372 is major, not critical — but it's a money leak). The criteria don't make the decision; they structure it, and they were written before anyone was tired or under pressure. That's their entire value, and it's why you wrote them in week one of the release instead of now.
Who Decides — and What You Own Instead
Here is the sentence to keep for your whole career: QA informs the release decision; the team makes it. The old fantasy of QA as gatekeeper — the tester who "signs off" and can block a release — died for good reasons: it made testers the single point of blame, turned every release into a QA-versus-everyone fight, and let the rest of the team outsource their judgment. Your actual power is different and, once you feel it, better: the clarity of the picture. Nobody in that room can weigh what you haven't shown them; a sharp, honest report steers decisions more reliably than any veto ever did. And shipping with known bugs? Normal — when chosen with open eyes. The low-priority corner of the severity 2×2 ships consciously all the time, with a known-issues list. The difference between a decision and an accident is that a decision was made — by people looking at your picture.
Friday, Resolved
So: Petra reads the one-pager. FW-372 is the sticking point — a money leak, but small per booking, and the hotfix is scheduled for Tuesday. She decides: ship, with FW-372 on the known-issues list and modification-heavy promo bookings monitored until the hotfix. You'd have held it; you say so — once, in one sentence, on the record: "for the record, I'd hold for the hotfix; the leak compounds with volume." Then you commit to the decision and help it succeed. That's not defeat; that's the role, played correctly — the forecaster reported the storm, the pilot chose the route, and both did their jobs. Next chapter, the toolkit widens: testing beneath the screens, where half of this release's bugs actually lived.
- "QA approves releases." QA informs them — and that's a feature of the role, not a demotion. The gatekeeper model made testers the single point of blame and let everyone else outsource judgment. Your power is the picture's clarity.
- "A bug found after release means testing failed." Testing samples risk (Principles 1 and 2). The professional question is whether the escape came from a conscious cut (on the record, weighable) or a blind spot (a lesson). The report's courage line exists exactly for this.
- "Good metrics are green metrics." A report that's all green and covers nothing is worse than red and honest. Decision-makers need the product's state, not a performance review of the testing.
- "Disagreeing with the ship decision means fighting it." One sentence, on the record, then commit and help it succeed. Re-litigating in the hallway spends credibility you'll want for the next storm report.
- The one-page release report is the highest-visibility artifact a QA engineer produces — it's read by everyone who matters, minutes before the biggest recurring decision the team makes.
- "What would you tell the team before release?" is a standard interview scenario, and the four-answer structure (covered/not, found, open, risks) is the model response.
- Role clarity — inform, don't decide — prevents the two classic career injuries: being blamed as the gatekeeper who "approved" a bad release, and burning trust by fighting decisions you don't own.
Knowledge Check
Petra asks "can we ship?" What shape does the professional answer take?
- "Yes" — the tests are done and mostly green
- "No" — there's still an open major bug
- An honest picture: coverage, findings, open bugs, and named remaining risks
- "That's not my department — ask the developers"
Why is "total test cases executed" a vanity metric?
- Because counting executed cases is technically difficult
- It measures activity, not the product's state — and rewards padding when targeted
- Because the count is usually calculated incorrectly
- Because no numeric metric can ever be useful
Why were the exit criteria written at planning time rather than debated on Friday?
- Standards set in calm resist the pressure that bends Friday negotiations
- Because criteria are legally binding once written
- So the release decision happens automatically without a meeting
- Because the sprint-planning tool has a mandatory field for them
Petra ships with FW-372 open; you would have held. What does the professional do?
- Say nothing — the decision is made and it's not your call
- State the dissent once, on the record, then commit and support the decision
- Escalate over Petra's head to force a hold
- Agree publicly but slow-walk the release checks
You got correct