Visual regression testing in GitHub pull requests

The workflow, start to finish

1. CI uploads screenshots. A step in your GitHub Actions workflow sends them, along with the pull request number.
2. Snapvisor compares. It finds the baseline for the pull request's base branch and diffs every screenshot.
3. The pull request shows the result. A commit status appears on the head commit and one comment lists the build.
4. A reviewer decides. They open the build, approve the intended changes and reject the regressions.
5. The status follows the decision. Approval moves it to success, rejection to failure.

Step 1: upload from GitHub Actions

Run your screenshot tests, then upload the folder they wrote. The upload step is one command:

- name: Upload screenshots to Snapvisor
  run: npx @snapvisor/cli upload ./screenshots
  env:
    ARGOS_TOKEN: ${{ secrets.ARGOS_TOKEN }}

On GitHub Actions the uploader reads the workflow's event payload, so it sends the pull request number and the pull request head commit without you passing them. Snapvisor uses the number to attach the build to the pull request, and reads the base branch from the pull request itself on GitHub.

You do not have to store ARGOS_TOKEN at all: the CI integration guide describes tokenless authentication for connected repositories, which only works for builds run in that repository's own GitHub Actions (not a fork). The ARGOS_* variable names are kept on purpose, so a pipeline already wired for the upstream SDKs works unchanged.

Step 2: which baseline the pull request is compared with

The base branch is the pull request's own base branch. Only if you set a reference branch on the upload does that win, and for a build with no pull request the project's default base branch is used. Snapvisor asks GitHub for the commit where the pull request branched off that branch and looks for a build on exactly that commit.

That lookup needs Snapvisor to read the repository through the GitHub App. With the light install, which reads no source, the CLI works the baseline out from your local git history instead.

• It must be an approved build. A build on that commit only counts as the baseline if it was approved, or if its pull request was merged. The exception is a base branch listed as auto-approved: there, merging is taken as the approval.
• No build on that commit. Snapvisor walks back through the commit's ancestors and uses the nearest build it finds. Builds with a standing rejection, subset builds and skipped builds are never used.
• No baseline found. The first build in a project has nothing to compare against, so there is no diff to review. As a last resort, a build is compared with an earlier approved build on the same commit and branch. The baseline guide has the full rules.

In practice that means running the same upload on pushes to your base branch as well as on pull requests, so each merge leaves a baseline for the pull requests that branch from it.

Step 3: what shows up on the pull request

Two things appear, and a third is optional.

• A commit status on the pull request's head commit, linked to the build. It is a GitHub commit status, not a check run. Its context is argos for a single build; the project name is added when another project is linked to the same repository, and the build name when the build is not called default.
• One comment per pull request. It is a table with a row per build: the status, an Inspect link and, when there are changes to review, a Review link. After reviews come in, the status text names the reviewers and shows the number of comment threads. The comment is updated in place on later pushes rather than posted again.
• A summary status (optional), for a commit that has several builds, such as separate Storybook and end-to-end runs. Its context is argos/summary and it rolls them into one result. In the project settings the mode is auto, always or never; set it to always if you want to make it a required check, as the summary checks guide explains.

The status moves through these states:

• Queued and in progress: pending.
• No changes found: success.
• Changes detected: failure, with the description "waiting for your decision". On an auto-approved branch the same result is a success.
• Approved: success.
• Rejected: failure.
• Expired because the screenshots never arrived: failure.

Changes detected map to a failure, so if you make the status a required check in GitHub branch protection, the pull request cannot merge until someone has made a decision.

Step 4: review, approve or reject

The reviewer follows the Review link in the comment (or the status link) to the build. From there:

• Start review opens the first changed screenshot. The shortcuts shown next to it are the arrow keys to navigate, y to approve and n to reject.
• Submit review opens a dialog with three choices: Approve, Reject or Comment, with an optional message. Comment gives general feedback without an explicit approval.

When the review is submitted, Snapvisor recomputes the build result and sends a fresh notification, so the commit status on the pull request updates. Reviews combine like this: only builds that detected changes need a review, each reviewer's latest review counts, a single rejection makes the build rejected, and otherwise a single approval makes it approved. A Comment review counts for neither side, and it replaces that reviewer's earlier approval or rejection.

That is what makes the approval the gate: the status that read "waiting for your decision" now reads "approved" and turns green.

Storybook and design systems

If your design system lives in Storybook, @snapvisor/storybook captures a screenshot of every story, through the Storybook Vitest addon or the Storybook Test Runner. Those screenshots are uploaded and reviewed on the pull request exactly like Playwright screenshots, so a change to a shared component shows up as a diff on every story it touches. See the Storybook section of the Percy migration guide for the setup.

npm install --save-dev @snapvisor/storybook

Limits to know about

• No GitHub App, no statuses. If the repository is not linked or the app has no installation, the build still runs and can be reviewed in Snapvisor; nothing is posted to GitHub.
• A deleted comment stays deleted. If someone deletes the Snapvisor comment, Snapvisor notices on its next update and does not post it again for that pull request. The commit status keeps updating.
• Merge queues. For builds from a merge queue the comment is not posted.
• Rejection wins. One reviewer's rejection keeps the status failed even if others approved, until that reviewer submits a new review.

Related reading

For the basics of the technique, read the visual regression testing guide. If you are weighing tools, the pages below set Snapvisor next to two of them.

Frequently asked questions

Sources

Review your first pull request