NightlyQAStatus
Guide · NightlyQA

Evidence-backed bug reports: what NightlyQA files and why

A bug report is only as useful as what it lets you do next. NightlyQA files each finding with enough evidence to reproduce it without asking anyone what they saw.

The evidence pack

Every failure writes six files:

  • Screenshot: a full-page capture at the moment of failure.
  • Video: a browser recording of the walk.
  • Console log: console errors, warnings and page errors.
  • Network log: failed requests, server errors and failed xhr/fetch calls.
  • DOM: the page HTML as the browser had it.
  • Repro steps: what to open or do to see the failure yourself.

The issue

The GitHub issue states what broke (workflow and step), the expected and actual result, the repro steps, links to the evidence, a severity label, when it was first seen, the run tag for any test data, and the environment (URL, browser version and commit). The redacted sample finding shows one, generated from a local fixture app.

Redaction

Console and network text is redacted before it is written. Query values for keys like token, password and secret become [redacted], and so does anything after Bearer. The evidence pack is meant to be shareable with the people fixing the bug.

One failure, one issue

Each finding has a stable signature built from the repo, workflow, step and error class. If the same failure happens again, NightlyQA adds a comment to the open issue instead of filing a new one. Local state backs this up, so a second issue is not filed even when GitHub search misses the first. When the failure stops, the issue can be closed automatically.

Safe by default

Issue filing is a dry-run unless it is turned on explicitly. A dry-run still writes the evidence pack and the local signature state, but makes no GitHub request.

NightlyQA is an early-stage product in portfolio dogfood. Public onboarding is not open yet. Request early access.