Topic 30

Test Execution: Runs, Statuses, the Daily Rhythm

Process

Design is thinking; execution is the contact with reality — running the checks, judging outcomes, and leaving a trail others can trust. It sounds like the mechanical part after the clever part. It isn't: execution is where every "hm, that's odd" either gets chased or lost forever, and where sloppy status-keeping quietly poisons the release decision three days later. This topic is the working discipline of the testing day.

The right mental model is a pilot's checklist discipline. The checklist was written in calm; flying it is the real event — where "that gauge reads odd" gets investigated and logged, or regretted. Same list every flight; full attention every flight anyway.

The Test Run

A test run is a set of test cases executed against one specific build in one specific environment — with the record to prove it: who ran what, when, on which build. You already know why the build matters (Chapter 2's argument-with-Omar lesson: results are meaningful only relative to a snapshot). The run record is that lesson institutionalized: when the release meeting asks "was the promo suite green?", the answer is "on build 4.2.18, staging, Wednesday — yes," and it means something. Runs live in the tracker or a test management tool, but a disciplined spreadsheet does the job at Fernway's size.

Statuses That Mean Something

Each case in a run ends in a status, and the vocabulary matters more than it looks. Pass — the expected result happened, exactly. Fail — it didn't; a verdict about the product. Blocked — the case couldn't be run at all: the environment is down, a dependency is missing, the feature it needs isn't deployed; a fact about testing logistics, not the product. Skipped — deliberately not run this time, per the plan. The distinctions carry real weight downstream: five fails means the product has problems; five blocked means you don't know — and "we don't know" and "it's broken" lead to completely different Friday decisions. Mislabeling blocked as fail (or worse, quietly as pass-later) is how release meetings end up reasoning from fiction.

Four statuses — what each one actually tells the team
Pass
behavior held, on this build
Fail
product problem → bug report
Blocked
couldn't run — an unknown, not a verdict
Skipped
chosen, planned, on record

Failing Well

A fail isn't the end of a row in a spreadsheet — it's the start of the machine you built across Chapters 3 and 6. The failed case births a bug report (the FW-312 anatomy), linked to the case (traceability), filed now — never batched to end-of-day, because details evaporate in hours: which build, what exact data, what you clicked between steps 3 and 4. When the fix arrives, the case gets re-run on the fixed build, and regression runs around it — the Chapter 3 loop in its natural habitat. One discipline makes all of it cheap: file at the moment of finding, while the scene is fresh and the evidence is one screenshot away.

Chasing the "Huh?"

Mid-case, running the promo suite, you notice something off to the side: the currency symbol flickered during page load. Not what the case checks; not a fail; probably nothing. This is the moment that separates execution from data entry. The undisciplined move is amnesia — finish the case, move on, forget. The disciplined move costs twenty seconds: a side-note — "currency symbol flickers on checkout load, build 4.2.18, saw it twice" — parked in your session notes, and a follow-up charter if it smells like smoke ("explore checkout rendering under slow connections, 30 minutes"). Half the interesting bugs in any product are found next to what someone was checking; the side-note habit is how they survive long enough to be chased.

The Rhythm of an Execution Day

Assembled, a realistic day mid-release at Fernway. Morning: smoke the new build — ten minutes, is 4.2.18 alive? Then the planned run: tier-1 promo cases, in order, statuses honestly recorded as you go. A case goes blocked — Paylane's sandbox is down again (of course it is); you mark it blocked, note the time, and switch to non-payment cases rather than idling — the run continues around the blockage. A case fails; you file the report now, with evidence, and keep moving. Standup or end of day, the run record reports itself: "promo run on 4.2.18: 14 pass, 1 fail — FW-372 filed, 3 blocked on the sandbox, resuming when it's back." One sentence, three statuses, zero fiction — and everyone who plans from it plans from the truth. That habit, repeated daily, is what execution discipline actually is.

Common Confusions
  • "Execution is just following the steps." Judgment lives in every outcome call, every blocked-vs-fail distinction, and every side-finding chased or lost. The steps are the score; execution is the performance.
  • "Blocked is basically a fail." A fail is a product verdict; blocked is a logistics fact — and they lead to different decisions. Five fails: the product has problems. Five blocked: you don't know yet, and pretending otherwise is fiction.
  • "File bug reports at the end of the day, in one batch." Details evaporate in hours — the exact data, the build, the click between steps. File at the moment of finding; the ten minutes now saves the cannot-reproduce ping-pong later.
  • "Side observations aren't my job mid-run." Half the interesting bugs live next to what you were checking. Twenty seconds of side-note keeps them alive; amnesia kills them permanently.
Why It Matters
  • Run records are the raw evidence behind Friday's release decision — status discipline today is decision quality in three days.
  • "14 pass, 1 fail, 3 blocked on the sandbox" is a sentence people can act on; "mostly done, some issues" is not. The vocabulary is the professionalism.
  • The side-note habit compounds into your error-guessing catalog (Chapter 4) — today's "huh?" is next quarter's ten-minute catch.

Knowledge Check

Why must a test run be tied to a specific build?

  • Because results are only meaningful relative to one specific snapshot of the code
  • So management can track how many builds testers process
  • Because test management tools require a build field
  • To prevent the same build from being tested twice

Paylane's sandbox goes down mid-run and three payment cases can't execute. What status do they get, and why does it matter?

  • Fail — the cases didn't produce their expected results
  • Blocked — they couldn't run, which is an unknown, not a verdict
  • Skipped — the tester chose not to run them today
  • Pass — the sandbox failure wasn't the product's fault

Why are bug reports filed at the moment of finding rather than batched at day's end?

  • Because trackers close for submissions in the evening
  • Details evaporate within hours, and the fresh scene makes the report reproducible
  • Because bugs filed earlier in the day get fixed the same day
  • Filing immediately is slower but looks more diligent

Mid-case, you notice the currency symbol flicker — unrelated to what the case checks. What's the disciplined move?

  • Stop the run and investigate the flicker immediately
  • Ignore it — it's not what the case is checking
  • Park a twenty-second side-note and queue a follow-up charter if it smells like smoke
  • File a full bug report for the flicker before continuing

You got correct