11.14 — Debugging failing tests
The defining scenario of the job: the suite is green on your machine and RED on CI — where there was no screen, no human, and no replay… except the evidence kit your config has been quietly collecting. Today: the trace — a flight recorder with time-travel — and the five-question method that turns any red into a diagnosis.
// red on CI — you weren't there. Evidence: // test-results/…/test-failed-1.png screenshot // test-results/…/video.webm the film // test-results/…/trace.zip flight recorder // $ npx playwright show-trace trace.zip // locally: time-travel debugging // $ npx playwright test --ui // mid-test breakpoint (8.3, browser edition): await page.pause(); // the policies that captured all this (11.3): // trace: "on-first-retry" // screenshot: "only-on-failure" // video: "retain-on-failure"
Set the scene honestly: 11.16’s robot ran your suite at 3am on a machine that no longer exists. One test is red. You cannot “just look” — there is nothing to look AT. Every debugging skill you have (8.3!) needs evidence to grip. Playwright’s answer: record everything worth gripping.
The deeper story, with the real names for things — this part is what turns “I saw it” into “I can explain it.”
Why trace: "on-first-retry" is the beloved default: recording costs a little speed, so the first (usually passing) attempt runs unrecorded — but when a test fails and retries (11.15), the RETRY records everything. You pay for evidence exactly when evidence is needed. Flip to "on" temporarily when hunting something rare.
The trace also captures test.step chapters (11.13) — a well-stepped long test reads as a table of contents in the viewer. This is where that hygiene habit pays its dividend: failures name chapters, chapters jump to snapshots.
The HTML report (11.2) embeds all of this: on CI, download the report artifact, open it, click the red test — screenshot, video, and trace are one click deep. 11.16 wires the upload; you already know how to read everything inside.
Job note: “walk me through debugging a CI-only failure” is THE senior-signal interview question in this field. You now own the complete answer: artifacts survive the machine (screenshot → right page?; trace → five questions: assertion, snapshot, locator, network, console), reproduce locally with --ui if needed, and the usual verdicts — stale auth bottle, backend 5xx, or a real race the trace timeline exposes. That answer, delivered calmly, reads as years of experience.
⌨️ read the flight recorder
The starter is a captured trace (as data). Write the diagnosis routine that walks the five-question method and names the true culprit — the way you will at work, weekly.
requirements:
- Keep the starter’s
traceobject: actions (with an ok flag), network entries, and console errors. - Question ①: find the first action where
okis false (4.10’s find) — printfailed at: NAME. - Question ④: find any network entry with status ≥ 500 — print
network: 500 from /api/search(built from the found entry). - Verdict: if a 5xx exists, the app wasn’t the bug’s author — print
diagnosis: backend failure, not a locator bug.
when you press RUN, the console must show exactly:
✏️ Quick check 1
Which artifact lets you inspect the page’s ACTUAL DOM as it was at each action — screenshot, video, or trace?
✏️ Quick check 2
The trace’s network tab shows /api/search answered 500 right before your assertion failed. Whose bug is it likely — your locator or the backend?
✏️ Quick check 3
Which command opens the local cockpit — watch mode fused with a live trace viewer?
🗣️ Now teach it back
Walk a friend through debugging a CI-only red: the evidence kit and what each piece answers, the trace’s three panes, the five-question method in order, and the two local tools.
Write it as if your friend is sitting next to you. Saved to your journal — future-you will use these notes to teach others.