Exploring Playwright end to end testing tool

First of all, in my human size tech team, we don’t have the resources to constantly test front-end application evolutions throughout the weekly features implementations. We are permanently searching for solutions that can guarantee a safer application for our users even if we don’t have plenty of QA testers. What if we implement end-to-end testing or try to?

What is end to end testing ?

End-to-end testing is a type of testing that verifies the entire or part of the software application in a specific scenario. It aims to ensure that features in application functions correctly and meets the user requirements.

In the JavaScript ecosystem, a cutting-edge front-end end-to-end testing tool for the modern web is Cypress.

He is the boss! Cypress rules the game.

Exploring end-to-end possibilies

During my tech searches for e2e testing best companions, even if Cypress rule the game, there is a single little problem with it.

We have a strong Safari browser usage in our users base.

Cypress implement recently a beta version of Webkit compatibility but it’s still experimental. There is my problem, if our features are not properly tested on a Safari mobile and desktop configuration, it ain’t solve our laking QA environnement.

During my search for a competitor in Cypress. I came across Playwright.

Their promise is attractive, an end-to-end cross-browser compatible testing tool with some resistance to flakyness.

Here is Playwright

For more details about what Playwright can do for you, I recommend this official documentation: Playwright documentation.

I had already tried Cypress test implementation in an previous React project and was particularly excited to try Playwright in my current Vue environment.

First i had to configure my Playwright.

playwright.config.ts

---
import { defineConfig, devices } from '@playwright/test'
import type { DeviceOptions } from './e2e/device-config'
import dotenv from 'dotenv'
---

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */

dotenv.config({ path: "./.env" });
/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig<DeviceOptions>({
  testDir: './e2e',
  outputDir: './e2e/test-results',
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: Boolean(process.env.CI),
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: 1,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [['html', { outputFolder: './e2e/playwright-report' }]],
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    baseURL: process.env.VITE_BASE_URL ,
    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry'
  },

  /* Configure projects for major browsers */
  projects: [
    // Setup project
    { name: 'setup', testMatch: /.*\.setup\.ts/ },
    {
      name: 'chromium',
      use: {
        ...devices['Desktop Chrome'],
        device: 'chromium'
      }
    },

    {
      name: 'firefox',
      use: {
        ...devices['Desktop Firefox'],
        device: 'firefox'
      }
    },

    {
      name: 'webkit',
      use: {
        ...devices['Desktop Safari'],
        device: 'safari'
      }
    },

    /* Test against mobile viewports. */
    {
      name: 'Mobile Chrome',
      use: {
        ...devices['Pixel 5'],
        device: 'pixel5'
      }
    },
    {
      name: 'Mobile Safari',
      use: {
        ...devices['iPhone 12'],
        device: 'iPhone12'
      }
    }
  ],

  /* Run your local dev server before starting the tests */
  webServer: {
    command: 'yarn dev',
    url: 'http://localhost:5173',
    reuseExistingServer: !process.env.CI,
    stdout: 'ignore',
    stderr: 'pipe'
  }
})

The most important thing is to adapt my configuration for the required browsers and devices.

The fact that it is able to test on Chrome, Firefox, Safari and those in mobile or desktop resolution perfectly matches the tests we want to guarantee on the feature we have put on production, even without a full QA team.

Terminal

npx playwright install
# Using Yarn
yarn playwright
# or
yarn playwright-ui

I particularly appreciated the UI mode which opens a browser window allowing precise control of the tests and visualization of the actions for each frame.

Playwright UI

Creating my first Playwright test

Here we go! There is nothing very difficult to understand at first sight, a javascript library using function to query part of DOM and trigger user action like. Discovering and wrighting my first test was pretty exciting but with implementations comes exceptions. Now we have new problems to solve.

My project is on onboarding funnel that keep progression section by section using API request and get a user onboarding state. We use a set of middlewares to redirect user managing between user access and funnel progression.

First of all, my user had to be logged to try a test on onboarding. I found a solution with beforeEach() function that give me a space to play a set of DOM queries before each of my devices tests.

onboarding.spec.ts

test.beforeEach(async ({ page, request }) => {

  //Start test by performing auth
  await page.goto(`${process.env.VITE_BASE_URL}`, {waitUntil: 'domcontentloaded' })

  const email = page.getByTestId('input-email')
  const password = page.getByTestId('input-password')
  const submitButton = page.getByTestId('button-login')
  await email.fill('w43eizqe@anonaddy.com')
  await password.fill('Password1!')
  await submitButton.click()

  // Make sure unboarded user is redirect to about onboarding page.
  await expect(page).toHaveURL(`${process.env.VITE_BASE_URL}/onboarding/about`)
})

After a successful first test, I have an error to all things done after. Using the UI Playwright frame mode, I can see that my tests were not on the same requirement than the first one. They are already filled! Indeed, my first test using API changing the user data, all the same tests but with a different browser or device are done in a new configuration.

I have to keep the initial user data state in a variable and restitute it to APIs with afterEach()

Get initial user

let user: any = undefined

test.beforeEach(async ({ page, request }) => {

  //Start test by performing auth
  await page.goto(`${process.env.VITE_BASE_URL}`, {waitUntil: 'domcontentloaded' })

  const email = page.getByTestId('input-email')
  const password = page.getByTestId('input-password')
  const submitButton = page.getByTestId('button-login')
  await email.fill('w43eizqe@anonaddy.com')
  await password.fill('Password1!')
  await submitButton.click()

  // Make sure unboarded user is redirect to about onboarding page.
  await expect(page).toHaveURL(`${process.env.VITE_BASE_URL}/onboarding/about`)

  const response = await request.get(`${process.env.VITE_API_URL}/me`, {
    headers: {
      'Content-Type': 'application/json; charset=UTF-8',
      Authorization: `Bearer ${token}`
    }
  })

  user = await response.json()
})

Reset user to initial test

test.afterEach(async ({ page, request }) => {
  if (user.data) {
    console.log('user after', user)
    await request.patch(`${process.env.VITE_API_URL}/me`, {
      data: { user: user.data },

      headers: {
        'Content-Type': 'application/json; charset=UTF-8',
        Authorization: `Bearer ${token}`
      }
    })
  }
})

There are a lot of changes I did because browsers don’t interact in the same way with DOM queries or user events. My biggest concern was with the Safari browser.

My feelings

I personally take a lot of time to enter in end-to-end but it is a good way to get my code better and understand all functionalities and usages of a feature.

Write end-to-end test is a long journey and it’s not done yet to me…