Custom Integrations

If Flakiness.io doesn’t have a first-party integration for your testing framework, you can still integrate with our platform using several approaches.

Integration Options

File an Issue

File an issue on our feedback repository to request official support for your testing framework.

Request New Integration →

JUnit XML

Many testing frameworks can output JUnit XML format, which Flakiness.io supports natively.

JUnit Integration →

Node.js SDK

Use our official SDK to build custom integrations in Node.js with full type safety and utilities.

Learn More ↓

Custom Reporter

Build your own reporter using the Flakiness Report specification and upload via CLI.

Learn More ↓

Node.js SDK

Installation

npm i @flakiness/sdk

Quick Start

import {
  FlakinessReport,
  GitWorktree,
  ReportUtils,
  writeReport,
  uploadReport,
  CIUtils
} from '@flakiness/sdk';

// Initialize git worktree and environment
const worktree = GitWorktree.create(process.cwd());
const env = ReportUtils.createEnvironment({ name: 'CI' });

// Create a simple test report
const report: FlakinessReport.Report = {
  category: 'testreport',
  commitId: worktree.headCommitId(),
  url: CIUtils.runUrl(),
  environments: [env],
  suites: [{
    title: 'My Test Suite',
    type: 'describe',
    tests: [{
      title: 'My Test',
      location: { file: 'test.spec.ts', line: 10, column: 1 },
      attempts: [{
        environmentIdx: 0,
        expectedStatus: 'passed',
        actualStatus: 'passed',
        duration: 100 as FlakinessReport.DurationMS,
      }],
    }],
  }],
  startTimestamp: Date.now() as FlakinessReport.UnixTimestampMS,
  duration: 100 as FlakinessReport.DurationMS,
};

// Write report to disk or upload to Flakiness.io
await writeReport(report, [], './flakiness-report');
// Or: await uploadReport(report, [], { flakinessAccessToken: 'your-token' });

The SDK provides:

Type-safe report creation with TypeScript types
Git integration for commit detection and path conversion
CI detection for automatic run URL and environment detection
Report utilities for environment creation, normalization, and manipulation
Upload/write functionality to save or upload reports

For more details, see the SDK repository.

Custom Reporter

If your testing framework doesn’t support JUnit XML output and you’re not using Node.js, you can create a custom reporter that generates Flakiness Reports manually.

Getting Started

Generate Report JSON

Review the Flakiness Report specification to understand the required JSON structure. Create a reporter plugin for your testing framework that outputs a report.json file following the specification.

report.json looks like this:

{
  "category": "bash",
  "commitId": "a1b2c3d4e5f6789012345678901234567890abcd",
  "environments": [{ "name": "ubuntu" }],
  "tests": [
    {
      "title": "My shell test",
      "location": {
        "file": "tests/shell.sh",
        "line": 3,
        "column": 1
      },
      "attempts": [
        {
          "environmentIdx": 0,
          "expectedStatus": "passed",
          "status": "passed",
          "startTimestamp": 1703001600000,
          "duration": 1500,
          "attachments": [{
            "name": "screenshot.png",
            "contentType": "image/png",
            "id": "5d41402abc4b2a76b9719d911017c592"
          }]
        }
      ]
    }
  ],
  "startTimestamp": 1703001600000,
  "duration": 2000
}

Include Attachments (Optional)

Place any test artifacts (screenshots, logs, videos) in the same directory as report.json:
```
 flakiness-report/
 ├── report.json
 └── attachments/
     └── 5d41402abc4b2a76b9719d911017c592
```
Upload to Flakiness.io

Use the Flakiness CLI to upload your generated report:
Terminal window
```
flakiness upload ./flakiness-report
```

Example Implementations

Looking for inspiration? Check out our existing integrations: