Getting Started

Flakiness.io is a test analytics service collects historical data about test runs and generates a report.

Flakiness.io seamlessly integrates with Playwright through our custom reporter. Available as an NPM package, the reporter automatically collects and uploads comprehensive test results data, including:

• Playwright Traces
• Screenshots
• Videos
• Console logs

Follow these steps to add Flakiness.io to your existing Playwright project:

1. Install the Flakiness.io reporter package:

   npm i @flakiness/report@latest

2. Configure the reporter in your playwright.config.ts file:

   playwright.config.ts

   import { defineConfig } from '@playwright/test';
   
   export default defineConfig({
     reporter: [
       ['list'],
       ['@flakiness/report/playwright-test'],
     ],
     retries: 1,
   });

3. Set up CI authentication

   Github Actions

   Add your Flakiness.io access token to your repository:

   1. Navigate to your GitHub repository settings
   2. Go to Secrets and Variables → Actions
   3. Add a new secret named FLAKINESS_ACCESS_TOKEN
   4. Set its value to the access token from your Flakiness.io project settings
   5. Pass secret as an environment variable to the npx playwright test step

      # In your GitHub workflow file (.github/workflows/playwright.yml)
      jobs:
        test:
          runs-on: ubuntu-latest
          steps:
            # ... other steps like checkout, setup Node.js, etc.
            - name: Run Playwright tests
              env:
                FLAKINESS_ACCESS_TOKEN: ${{ secrets.FLAKINESS_ACCESS_TOKEN }}
              run: npx playwright test

After completing these steps, your Playwright tests will automatically upload results to Flakiness.io. You can view your test analytics and insights in the Flakiness.io dashboard.