In this article, I’ll explain how to integrate Playwright tests into the deployment process of your Next.js site hosted on Vercel and how to use them to protect the main production branch.

What is Playwright?

Playwright is a testing framework that allows us to run end-to-end tests. The tutorial by Next.js explains the setup and usage well, so let me just quickly summarize the steps:

1. Install Playwright:
   npm i @playwright/test --save-dev
2. Add test script into package.json:
   "test:e2e": "playwright test"
3. Create your tests (see below) and run them:
   npm run test:e2e

To illustrate what the tests look like, see one that we use on our own site:

import { test, expect } from '@playwright/test'

test('Syntax LP adds UTM parameters into cookies', async ({ page, context }) => {
	await page.goto('/syntax')
	const cookies = await context.cookies()
	await expect(cookies.find(c => c.name == 'utm_source').value).toBe('syntaxfm')
})

This test checks that whenever you reach our https://kontent.ai/syntax page, you get a utm_source cookie with the value of syntaxfm. It’s a functional cookie, and it ensures that Syntax.fm podcast listeners get the promised special offer.

We can run the tests manually with the npm script mentioned earlier:

npm run test:e2e

But we aim to run these tests automatically on every pull request to give us confidence that the website is always working and that the new code won’t break essential parts of our site.

GitHub and Vercel pipeline

But first things first. Let’s start with creating the site on Vercel. The platform always prompts you to establish a connection with the relevant GitHub repository:

The Vercel for Git integration then automatically deploys a preview of your site on every branch push. This is a required prerequisite to any integration and end-to-end test. You always need to have a deployed site to run your tests against.

So, when you push into any branch, Vercel will try to build your site:

These previews are nice but can often fail because even though you pushed some changes, you’re still actively working on that code. We actually want to run the automated tests only when we need to merge our new code into production—in other words, on every pull request.

Adding Playwright tests into the GitHub pipeline

Now, as I mentioned above, we need to have a built site before we can run the tests. We could, of course, use a GitHub Action to build the site on every PR, but:

• We’d still need to deploy it somewhere so that Playwright can access it.
• It would be a waste of resources, as Vercel is already building and hosting the site for us.

Instead, we’ll just wait for Vercel to build the preview and use it for our tests. We’ll create a GitHub Action that will:

• Wait for Vercel and grab the preview URL
• Set up Playwright environment
• Run the tests

Setting up the GitHub Action trigger

Now, for our GitHub Action, we need a trigger. We could use the issue_comment trigger, as Vercel always comments on the pull request with the URL when the preview is ready. The problem is, GitHub won’t let us configure the GitHub Action using that trigger as a required status check of PRs:

That effectively prevents it from blocking the merge in case some tests are failing. So not good enough for our use case. However, it’s OK to use the issue_comment trigger for non-blocking tasks like fetching the Lighthouse scores.

In our case, we need to use the pull_request trigger:

name: Deployment tests
on:
  pull_request:
    branches:
      - main

This example also limits the trigger to the main branch as we only want to run the tests on PRs in the production branch.

Fetching the Preview URL from Vercel

Next, we’ll wait for Vercel to deploy the site and fetch the preview URL from its comment:

jobs:
  test_setup:
    name: Test setup
    runs-on: ubuntu-latest
    outputs:
      preview_url: ${{ steps.waitForVercelPreviewDeployment.outputs.url }}
    steps:
      - name: Wait for Vercel preview deployment to be ready
        uses: patrickedqvist/wait-for-vercel-preview@1.2.0
        id: waitForVercelPreviewDeployment
        with:
          token: ${{ secrets.GITHUB_TOKEN }}
          max_timeout: 600

Note: You don’t need to set the GITHUB_TOKEN, it’s injected by GitHub automatically.

We’re using the Wait For Vercel Preview GitHub Action to do the waiting for us. It periodically checks the issue comments and outputs the preview URL when the deployment is ready.

Note: You can define additional params like the check frequency or Vercel password if needed.

The downside here is that GitHub Action actively waits on Vercel and thus keeps consuming your GitHub Action minutes, so make sure:

• Your site build is optimized (I know, so at least reasonably optimized :-).
• The GitHub Action has a timeout set—the timeout in the example is set to 10 minutes.
• The GitHub Action is triggered only when needed—the trigger in the example is set to the main branch only.

 When we get the preview URL, we can continue with the next step.

Running Playwright tests using GitHub Action

Now is the time to prepare the environment and execute the tests. The process is well described in the Playwright docs. You need to:

• Install dependencies
  npm ci
• Install Playwright
  npx playwright install --with-deps
• Run tests from your solution (make sure to use the correct script name from your package.json)
  npm run test:e2e

jobs:
  ...
  test_e2e:
    needs: tests_setup
    name: Playwright tests
    timeout-minutes: 5
    runs-on: ubuntu-latest
    steps:
      - name: Prepare testing env
        uses: actions/checkout@v2
      - uses: actions/setup-node@v2
        with
          node-version: "14"
      - run: npm ci
      - run: npx playwright install --with-deps
      - name: Run tests
        run: npm run test:e2e
        env:
          PLAYWRIGHT_TEST_BASE_URL: ${{ needs.test_setup.outputs.preview_url }}

A very useful addition is the environment variable PLAYWRIGHT_TEST_BASE_URL that lets you run the tests on both local and in the CI on preview deployments. You can feed it into Playwright by creating a playwright.config.ts file at the root of your project:

import { PlaywrightTestConfig } from '@playwright/test'
const config: PlaywrightTestConfig = {
    use: {
        baseURL: process.env.PLAYWRIGHT_TEST_BASE_URL || 'http://localhost:3000'
    }
}
export default config

See the full GitHub Action code on GitHub.

Setting the GitHub Action as a required check

The last step is to set the created GitHub Action as a required status check. You can do that this way:

• Go into Settings of your repository
• Select Branches
• Edit an existing protection rule or create a new one for your main branch
• Check Require status checks to pass before merging
• Find and select the Playwright tests GitHub Action using the search field

Note: If you can’t find the action using the list, make sure you’re using the pull_request trigger in the action implementation and that it already ran at least once.

Make sure to save changes at the bottom, and you’re done. Congratulations. You’ve successfully protected your production branch with Playwright and E2E tests.

Conclusion

In this article, I showed you how to run Playwright end-to-end tests on every pull request that wants to merge new code into production. The example above is tailored to Vercel deployments, but you can easily adjust it for any other provider. The end-to-end tests complement unit and UI tests and provide an additional level of safety to your websites. As a result, you can feel more confident even about your Friday deployments :-).

We’re using Playwright on multiple projects ourselves. If you need a hand or want to discuss your projects, join our Discord.