Playwright Visual Regression

Takes two runs of visual-snapshot's capture — a baseline and a current one — and finds the pages that actually changed, filtering out the noise that's normal (anti-aliasing, a carousel that rotated) from the noise that isn't (a layout that shifted, a section that disappeared).

Relationship to visual-snapshot

This skill doesn't capture anything new on its own — it consumes two capture runs visual-snapshot already produced (same viewport, same settle discipline, same snapshots/<date>/<page-slug>.png naming). Consistency between the two captures is what makes a diff meaningful at all: if the baseline was taken at a different viewport or without letting the page settle, the diff will be dominated by capture-condition noise that has nothing to do with real changes. If there's no baseline yet, run visual-snapshot and treat its output as the baseline — that's a legitimate starting state, not a missing prerequisite to work around.

Core principles (and why)

Two thresholds, not one. A per-pixel color-difference threshold (commonly ~0.1) tolerates ordinary anti-aliasing and font-rendering noise at the level of individual pixels. A separate page-level tolerance — a maximum acceptable percentage of changed pixels (commonly ~1%) — tolerates the fact that a handful of genuinely different pixels scattered around shouldn't fail a whole page, while a large contiguous change should. Relying on just one of these either misses real regressions or drowns in false positives from harmless rendering noise.

Mask known-dynamic regions before diffing — don't just note them and move on. visual-snapshot already flags carousels, timestamps, and similar widgets as "captured in an arbitrary state, this is normal." In a regression check, that same region will register as "changed" on essentially every single run unless it's explicitly excluded from the pixel comparison. Masking it out entirely is the difference between a report someone trusts and one they start ignoring because it cries wolf every time.

A flagged diff never gets silently promoted to the new baseline. Baseline management, not the diffing itself, is the part of visual regression testing that actually causes real-world setups to fail — because applications legitimately change, baselines have to update, but only deliberately. Every flagged page needs an explicit human decision: accept as an intentional change and update the baseline, or treat it as a real regression and fix the underlying issue. Never auto-accept the current run as the new baseline just because it's newer — that's exactly how a real regression gets permanently baked in as "correct."

Prefer @playwright/test's native toHaveScreenshot() if a real Playwright Test project already exists — it gets you more than pixelmatch alone. It automatically disables CSS animations and waits for fonts to load before capturing, which removes a whole category of false positives from animation frames or font-swap timing that a bare pixelmatch comparison would still be exposed to. For an ad hoc comparison of two independent visual-snapshot capture runs — the more common case in this toolkit, since not every project has a full Playwright Test setup — use pixelmatch directly against the two PNG sets instead.

Some pages cannot be pixel-diffed at all — exclude them deliberately rather than tuning thresholds until they pass. Canvas/SVG charts, waveform renderers and anything drawing from resampled data can re-render with sub-pixel variation on every load even when the underlying data is byte-identical. Masking the chart region helps but often leaves the page over tolerance anyway. When you hit one, establish it by evidence — try the font wait, try a mask, try a longer settle — and if it still fails, exclude the page and say what you tried. Then cover it functionally instead: asserting a waveform renders a plausible heart rate and a non-trivial path count is a stronger check than asserting its pixels match yesterday's. Quietly raising a page's tolerance until a genuinely unstable page goes green is how a suite stops detecting anything.