Automation

Visual regression testing: when the test passes but the screen is broken

The E2E test clicks the button, checks that the right text appeared and goes green. Meanwhile, in production, broken CSS pushed that button off the screen, made the text white on a white background or collapsed the whole menu. The test passed because it verifies the DOM, not the appearance. That gap — functional ✓, visual ✗ — is exactly what visual regression testing exists to close.

The blind spot of functional tests

A traditional test asks "does the element exist and have the right content?". It can assert that <button>Comprar</button> is in the DOM and clickable. What it does not see: that the button's color equals its background, that a z-index hid it behind another element, that the font didn't load, or that at 1280px wide the card spilled out of its container. For the user, that's a broken product. For the functional suite, it's all green.

How it works: baseline, comparison and diff

The idea is straightforward: take a picture of the screen and compare it with a previously approved picture. The flow has three steps:

  1. BASELINE      captures the "correct" screen and stores it as a reference
        │
  2. COMPARISON    on every run, takes a new screenshot and compares it
        │          pixel by pixel with the baseline
        ▼
  3. DIFF          ┌─ identical → passes
                   └─ different → fails + image highlighting what changed

When there is a difference, the tool generates a diff image highlighting the changed regions. Then comes the human decision: if the change is a bug, you fix it; if it's an intentional change (you really did redesign the button), you approve the new baseline and it becomes the reference from then on.

The big enemy: the false positive

Comparing pixel by pixel is powerful and naive at the same time. A lot of things change the image without anything being broken — and each one becomes a red test that isn't a bug:

Source of noiseHow to tame it
Font anti-aliasing across OSesGenerate the baseline in the same environment (container/CI), not on your laptop
Dynamic content (dates, names, ads)Mask/ignore the region in the screenshot
Animations and carouselsDisable animations and wait for the screen to settle
Small rendering differencesSet a tolerance threshold (e.g. 0.1%)
Visual testing without false-positive control dies in two weeks: the team gets tired of approving diffs that aren't bugs and starts clicking "approve all" — the moment the test stops testing. Baseline stability isn't a detail, it's what keeps the test alive.

Pixel by pixel vs. perceptual comparison

Newer tools don't compare raw pixels: they use a perceptual diff, which ignores variations imperceptible to the human eye (like a minimal anti-aliasing difference) and focuses on what a person would actually notice. It's the difference between a test that complains about everything and one that complains about what matters. Common tools: Percy, Chromatic (for Storybook), Applitools (with AI-based checking) and the native visual snapshots of Playwright (toHaveScreenshot) and Cypress.

// Native visual snapshot in Playwright
await expect(page).toHaveScreenshot('checkout.png', {
  maxDiffPixelRatio: 0.01,        // tolerance
  mask: [page.locator('.data-hoje')] // ignores a dynamic region
})

Where to fit it (without clogging the pipeline)

Visual regression is expensive: it generates images, requires storage and demands human review of the diffs. So don't go around screenshotting everything. The strategy that works is to aim at the points of highest visual risk — the design system / component library (where one change contaminates the whole app) and a handful of key pages, at a few representative screen widths. It's the pyramid logic again: many cheap tests at the bottom, few expensive visual tests at the top.

In the pipeline, the natural place is the pull request: the tool posts the diffs as a check, and the merge waits for someone to approve the visual change. That way appearance review happens alongside code review, and not in the user's complaint after the deploy.

Conclusion

Functional testing guarantees the system does the right thing; visual regression testing guarantees it looks right — and the two fail for different reasons. Capture baselines in a stable environment, attack false positives with masks, thresholds and perceptual diff, and concentrate screenshots on the design system and critical pages. Done this way, it catches the whole class of bugs that slips through between "the test is green" and "why does the site look like this?".

Read also

Back to the blog