Mike Stop Continues

Configuring Snapshot Tests in Playwright

Mike Stop Continues's photo
Mike Stop Continues
·

4 min read

Playwright's snapshot tests are incredibly powerful, but they can be very flaky without the right configuration.

By the end of this article, you'll have a solid understanding of how to configure your visual tests for the perfect balance of stability, security, and validity.

If you're interested in going deeper into visual testing, you'll love my Ultimate Guide to Visual Testing in Playwright. It covers advanced testing techniques, CI/CD integration, and more.

Let's get started...

Custom Snapshot File Names

Playwright automatically names your snapshots based on the name of the test. If you're only using snapshots for visual testing, that's fine.

However, many users like to repurpose these images for documentation, style guides, and deployment reports. These tasks demand consistent file names.

Name your snapshots like so:

test('custom snapshot names', async ({page}) => {
  await page.goto('https://www.browsercat.com');
  const $hero = page.locator('main > header');

  await expect(page).toHaveScreenshot('home-page.png');
  await expect($hero).toHaveScreenshot('home-hero.png');

  const $footer = page.locator('body > footer');
  const footImg = await $footer.screenshot();

  expect(footImg).toMatchSnapshot('home-foot.png')
});

Note: Custom snapshot names don't completely control the file name unless you also configure custom directories. With the default configuration, Playwright adds a suffix to your file names to ensure they're unique across projects.

Read on for how to customize the snapshot directory structure...

Custom Snapshot Directories

By default, Playwright stores snapshots in the same directory as the test file that created them. This method has numerous shortcomings:

  • It makes navigating your codebase more difficult.

  • It's hard to exclude snapshots from version control.

  • It's not easy to cache results between CI/CD runs.

  • And it gets in the way of custom snapshot file names.

So let's tell Playwright to store our snapshots in a custom directory. Update your playwright.config.ts:

export default defineConfig({
  snapshotPathTemplate: '.test/snaps/{projectName}/{testFilePath}/{arg}{ext}',

  // etc.
});

With the above configuration, if our test file is stored at ./tests/homepage.spec.ts, the Playwright will store our snapshots like so:

.test/
  snaps/
    tests/
      homepage.spec.ts/
        home-page.png
        home-hero.png
        home-foot.png

With this pattern, you can exclude your snapshots from version control, you can cache your results in CI/CD, and you can easily review failed tests as they arise.

Read more about the your options in the Playwright docs for snapshot path templates.

Make Visual Tests More Forgiving

Out of the box, visual tests are very strict. If a single pixel fails, your test fails. Thankfully, Playwright provides numerous controls for tuning how sensitive your visual tests should be.

Here are your options:

  • threshold: How much must a single pixel vary for it to be considered different. Values are a percentage from 0 to 1, with 0.2 as the default.

  • maxDiffPixels: The maximum number of pixels that can differ while still passing the test. By default, this option is disabled.

  • maxDiffPixelRatio: The maximum percentage of pixels that can differ while still passing the test. Values are a percentage from 0 to 1, but this control is disabled by default.

You can tune these options globally or per-assertion.

Configure forgiveness globally

To configure your snapshot options globally, update your playwright.config.ts:

export default defineConfig({
  expect: {
    toHaveScreenshot: {
      threshold: 0.25,
      maxDiffPixelRatio: 0.025,
      maxDiffPixels: 25,
    },
    toMatchSnapshot: {
      threshold: 0.25,
      maxDiffPixelRatio: 0.025,
      maxDiffPixels: 25,
    }
  },
});

Override forgiveness per-assertion

To override your global snapshot options, update your test file:

test('forgiving snapshots', async ({page}) => {
  await page.goto('https://www.browsercat.com');
  const $hero = page.locator('main > header');

  await expect(page).toHaveScreenshot({
    maxDiffPixelRatio: 0.15,
  });

  await expect($hero).toHaveScreenshot({
    maxDiffPixels: 100,
  });
});

How do I tune these options?

To converge on the correct setting for your app, review the failing "diff" images produced by your tests. In those images, yellow pixels indicate pixels that are within the threshold allowance, while red pixels fail the test.

First try to increase threshold, and see if you can gobble up the red pixels in your snapshots. But be careful: a high threshold will cause false negatives. I wouldn't go any higher than 0.35, and that's high.

If threshold doesn't work, focus on maxDiffPixelRatio and maxDiffPixels. And consider the trade-offs these options entail.

Since maxDiffPixelRatio is relative to image size, it's more likely to yeild good results across a wide range of images. This makes it a good contender for a global setting... but only if you set it conservatively! After all, 10% of a full page image will allow a lot of variation.

On the other hand, since maxDiffPixels is a constant value, it gives you much better control on individual tests. But it's also much more risky if applied globally, even at a low setting. For small images, a high maxDiffPixels might constitute a huge percentage of the image.

Next Steps...

That's really all there is to configuring your visual tests, but there's so much more to learn. For advance snapshot testing strategies and ready-made solution for running visual tests in CI/CD, check out my Ultimate Guide to Visual Testing in Playwright.

In the meantime, happy testing!

playwrightautomationTestingJavaScriptuser experience