Playwright

Demo

Report demo is available at https://flakiness.io/flakiness/demo-attachments

The official Flakiness.io reporter for Playwright Test.

Installation

To install Flakiness.io Playwright Test reporter, follow these steps:

1. Install the Flakiness.io SDK package:

   npm i @flakiness/sdk@latest

2. Configure the reporter in your playwright.config.ts file:

   playwright.config.ts

   import { defineConfig } from '@playwright/test';
   
   export default defineConfig({
     reporter: [
       ['@flakiness/sdk/playwright-test'],
     ],
   });

CI/CD Integration

To upload reports, you need your project’s Access Token. You can find this in your project settings on flakiness.io.

When the Access Token is provided, the reporter will upload the report to Flakiness.io in the end of the run. You will see a confirmation in your terminal:

  45 passed (1.5m)
[flakiness.io] ✓ Report uploaded
[flakiness.io] https://flakiness.io/flakiness/flakiness/run/6574

GitHub Actions

Add your Flakiness.io access token to your repository:

1. Navigate to your GitHub repository settings
2. Go to Secrets and Variables → Actions
3. Add a new secret named FLAKINESS_ACCESS_TOKEN
4. Set its value to the access token from your Flakiness.io project settings
5. Pass the secret as an environment variable to the npx playwright test step

# In your GitHub workflow file (.github/workflows/playwright.yml)
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      # ... other steps like checkout, setup Node.js, etc.
      - name: Run Playwright tests
        env:
          FLAKINESS_ACCESS_TOKEN: ${{ secrets.FLAKINESS_ACCESS_TOKEN }}
        run: npx playwright test

Usage

Once installed, every test run will produce a flakiness-report folder in the current working directory. This folder will contain a report.json file along with all the attachments.

Tip

Make sure to add flakiness-report folder to your .gitignore.

To open the report, use flakiness CLI tool:

flakiness show
# optionally pass a path to flakiness-report folder
flakiness show ./flakiness-report

Tip

To see a cloud history in the local report, make sure to link it to a remote project. See CLI Tool for details.

Environment Configuration

By default, the Flakiness.io Playwright Test reporter collects information about the operating system, Playwright Project configuration path, and project name.

You can also define custom key-value pairs using the FK_ENV_* environment variables:

export FK_ENV_ENDPOINT=staging
export FK_ENV_DB_PROVIDER=mysql