Works / QAWebLedger · Post-Deploy QA Checklist

The Checklist That Catches What the Diff Doesn't

Every release now arrives with its own QA list: the pages it touched, the CMS edits a diff misses, and what to do about each. Every one logged to a searchable changelog.

Client: Dev + Marketing · Role: Automation Engineer · Year: · Stack: n8n · GitLab · WordPress · Slack

The story

The before: Working out a release's blast radius meant reading a diff and guessing which pages it touched, so QA re-tested by instinct. CMS edits never showed up at all.

The turn: So I built QAWebLedger. Every deploy reads the code range and the CMS, reconciles them, and posts one checklist — a line per page, each with its action, each logged for later.

How it works

  1. GitLab: Each release is compared against the last one actually deployed.
  2. Classify: Around 22 ordered rules place every changed file in one tier — specific page, new page, flagged for removal, site-wide, infrastructure, triage. The tier is the action you're asked to take, and anything that can't be pinned to exactly one page goes to a person, never a guess.
  3. WordPress: The live template list is read from the site, not inferred from folder names — the handover said one folder, it's five, and about 159 templates would have been missed.
  4. Slack + the changelog: The checklist goes to whoever shipped; a row per page to a month-by-month log marketing's Monday digest reads off. One template drives 1,321 pages, so bulk edits collapse to a line while new pages stay itemised.

Detecting change is the easy half. The hard half is deciding which pages a person actually has to open and check — miss one and a broken page ships, flag everything and the list stops being read. So impact is judged by what a file does, not where it sits: a file nothing references yet is ignored, one that loads globally is flagged everywhere, and anything ambiguous is treated as the riskier case.

What changed

which pages do we QA on this one? did anything change in the CMS? which pages went live last month?

The numbers: ~2,100 — pages watched every release, code and CMS both.

A Slack 'Deployment Complete' message listing the pages to check after that release, grouped by what to do, with a link to the webpage changelog
The checklist, as it lands in Slack.

Fair questions

what if someone edits in the CMS and never deploys?

It surfaces on the next release — the trigger is always a code deploy.

what if a template ships before the page exists?

The row stays open and resolves on a later deploy. If the page never appears, it retires after 30 days.

does this only work on WordPress?

No. The same design now runs against a second site on a different stack, where the CMS half doesn't exist.

The reel continues

Previous: MARA · Tickets Reporter · Next: EmployeeCare · Leave Request Automation · open my notebook?

Still deciding what to QA after a deploy by hand, by instinct, after the fact? Check It — or discover my mastery.