Topic 24

Test Data & Traceability

Documentation

Every test needs stuff to test with: a traveler account, a booking in the right state, a property with exactly one room left, a promo code that's active. Somebody has to make that stuff exist — and nobody warns juniors how much of real testing life is exactly this. Test data is the unglamorous discipline that separates smooth testing from daily improvisation; traceability is its paperwork twin, the thread that ties every test back to why it exists. One page for both, because both answer the same question from different ends: is everything my testing needs actually in place?

Film sets solved this decades ago: the props department. The courtroom scene can't shoot if someone forgot the murder weapon — so props are listed against scenes, prepared in advance, and checked before the cameras roll. Your test data is props; traceability is the list against scenes.

Test Data Is Designed, Not Found

Here's the connection beginners miss: your data requirements come straight out of Chapter 4. The boundary tests need bookings of exactly 1, 3, and 30 nights. The decision table's column 1 needs a member account with an active SUMMER15. The state diagram needs a booking sitting in every state — including one frozen at pending, which doesn't occur naturally for longer than a few minutes. None of this data is lying around waiting; it's manufactured to specification, and the specification is your test design. Data is the technique made physical — which is why "I'll test with whatever's in staging" is the data-layer version of "I'll click around and see."

The Canonical Dataset

The professional move is a canonical dataset: fixed, named fixtures that staging always contains. Fernway's, after your first month: test travelers with known passwords and known member status; The Juniper Inn with known rooms at $100 a night (TC-118's arithmetic depends on that number); Saltwater Lodge configured for edge cases — one room left, seasonal pricing; SUMMER15 permanently active. Stable names buy three things. Reproducibility — TC-118 produces $255.00 on every run because the price is pinned. Readable bug reports — "book Juniper for 3 nights as traveler-basic" is a reproduction step anyone can follow. And speed — no twenty-minute setup safari before each test session.

The thread — one story, its tests, its bug
Story FW-210the 48-hour cancellation rule
TC-131…134boundaries · states · refund math
Data to specbookings at 47h / 48h / 49h
Bug FW-329linked back up the chain

Data Hygiene: the Three Classic Failures

Shared test data fails in three repeating ways, worth knowing by name before they happen to you. Tests that eat their data — Saltwater Lodge's last room, once booked by your test, is gone; the next run of the same test fails for a reason that has nothing to do with the product. Consumable data needs resetting or re-creating per run. Tests that collide — you and a future colleague both testing as traveler-basic at once, each corrupting the other's run (the environment lesson from Chapter 2, now at the data layer). Shared accounts need either coordination or separation. Stale data — the product changed, old fixtures didn't; half of staging's bookings now lack a field the new code requires, and everything touching them misbehaves in ways no requirement explains. When staging acts haunted, check the data's age before the product's code.

Never Real User Data

One rule in this book gets stated without a softening clause: production data does not go into test environments. Real travelers' names, emails, bookings, and payment details are private; copying them to staging — where the whole team has access, where bug report screenshots get attached, where security is looser by design — is a privacy violation and, in most places, a legal one. "But real data is more realistic" is true and irrelevant: realism comes from designing data with real-world messiness (long names, apostrophes — O'Brien from the catalog, unicode, boundary dates), not from borrowing actual humans. Anonymization tools exist — named here so you know the word — but the junior rule is simpler and safer: test data is made, never copied.

Traceability: the Thread Both Ways

Traceability is the linked chain: requirement → test cases → bug reports. Its everyday form is humble — the "linked story" field on TC-131, the "found by TC-133" link on bug FW-329 — but it answers two questions that otherwise cost an afternoon. Forward: "which tests cover the cancellation story?" — Nadia asks exactly this before the release, and the answer is a filter, not a shrug. Backward: "this test is failing — which requirement, and how much should we care?" — the link tells you what's actually threatened. The heavyweight version (a full traceability matrix, every requirement × every test in a grid) lives in regulated industries; Fernway's version is tracker links, maintained as you go. Same idea, right-sized — by now you know that song.

And the practical punchline for the 48-hour data problem: bookings created 47, 48, and 49 hours before a check-in can't be conjured by waiting around. You ask Omar, and he builds a staging helper that creates back-dated bookings on demand — your data problem becomes a one-click fixture. That's the mature pattern: test data needs become tooling requests, and developers are usually happy to build them once they see the alternative is you waiting 47 hours between test runs.

Common Confusions
  • "Test data is whatever happens to be in the environment." Unowned data rots, collides, and gets eaten. Canonical fixtures with stable names are owned, reset, and reproducible — the difference between a props department and a junk drawer.
  • "Production data is more realistic, so it's better." It's a privacy violation with legal teeth. Realism is designed in — messy names, boundary dates, unicode — without borrowing actual humans. Made, never copied.
  • "Traceability is enterprise bureaucracy." A tracker link from test to story is traceability. The ceremony scales with the stakes; the two questions it answers — what covers this? what does this failure threaten? — matter on every team.
  • "Weird staging behavior means the product broke." Check the data's age first: stale fixtures after a product change produce hauntings no requirement explains. Data hygiene failures impersonate product bugs constantly.
Why It Matters
  • The test data problem is a daily junior reality nobody warns you about — knowing the canonical-dataset pattern and the three hygiene failures saves weeks of accumulated friction.
  • The never-real-data rule protects you from the one mistake in this chapter with legal consequences; it's also a standard compliance interview question.
  • Traceability turns Nadia's "are we covered on cancellations?" into a link instead of a shrug — and links you maintain as you go cost seconds, while reconstructing them later costs afternoons.

Knowledge Check

Where do test data requirements come from?

  • From the test design — each technique's cases specify the data they need
  • From whatever already exists in the staging environment
  • From a copy of the production database
  • From the developers, who decide what testers may use

A test passes today and fails tomorrow with no product change. The test books Saltwater Lodge's last available room. What's the likely cause?

  • A regression introduced overnight by the developers
  • The test consumed its own data — yesterday's run took the last room
  • Paylane's sandbox went down again
  • The expected result was written incorrectly

Why is copying production data into staging forbidden?

  • Because production databases are too large to copy
  • Real users' private data would spread into a loosely secured, widely shared environment
  • Because real data is too realistic and would confuse the tests
  • Because staging environments can't technically hold production data

What question does forward traceability answer?

  • "This test failed — which requirement does that threaten?"
  • "Which tests cover this story?"
  • "Which environment and build was this test run against?"
  • "Which tester executed this case last week?"

You got correct