Test

The Test is the fundamental primitive in Flakiness.io. It represents a single, atomic unit of execution within your codebase that produces pass, fail, or skip result.

Test Identity

To track stability over time, Flakiness.io must uniquely identify each test across different commits and branches. A test’s unique identity is generated from a combination of:

1. Location: The relative file path within the git repository, starting from the repository root.
2. Hierarchy: The parent suites containing the test.
3. Title: The specific name of the test case.

These attributes combine to form the internal identifier for the test.

Duplicate Tests

If multiple tests exist in the same file, with the exact same Suite Hierarchy and Title, Flakiness.io cannot distinguish between them. In this scenario, their run results will be merged together into a single entry in the report.

History & Refactoring

Because a test’s identity is strictly tied to its name and structure, changes to your code can affect how Flakiness.io tracks its history.

Renaming Tests

If you rename a test or change its parent suite names, Flakiness.io sees this as a new test.

• Consequence: The history is “busted.” The old test (with the old name) stops receiving updates, and a new history timeline begins for the new name.
• Previous History: The old history remains accessible but will not be linked to the new test.

Moving Tests

Moving a test’s position within the same file (e.g., changing line numbers or reordering tests) does not affect its history.

• Consequence: Flakiness.io correctly identifies it as the same test, and its history continues uninterrupted.