Spec Files Explorer

Spec-level Health and Performance Dashboard - Flakiness, Failure Rate, Duration, Suite Size

The Spec Files Explorer allows users to track and analyze spec files performance using various metrics such as:

• Duration

• Failure Rate

• Flakiness Rate

• Timeout Rate

• Suite Size

Spec Files Explorer health and performance dashboard - flakiness, failure rate, duration, suite size.

Currents calculates the metrics based on the executions recorded during the provided date range and filters. A typical use case is to monitor the performance of the spec files that cause the degraded performance of the overall test suite - e.g. spec files with the highest flakiness rate, longest duration, etc.

Spec Files Explorer Metrics

Average Duration

The average duration of fully completed spec files, excluding timed-out or cancelled spec files. You can exclude or include failed executions for calculating the average duration in Settings (see below).

Overall Executions

How many executions were included during in the selected period and match the filters.

Failure Rate

The failure rate is the percentage of spec file executions that failed. A spec file is failed if it had 1 or more failed tests. See Spec Files Status.

Timeout Rate

The percentage of spec file executions that timed out. A spec file is marked as timed out if we didn't report its test results before exceeding its run's timeout. See Run Timeouts.

Flakiness Rate

The percentage of spec file executions that had 1 or more flaky tests. See Flaky Tests.

Suite Size

The number of tests in a spec file (including skipped tests).

Only fully completed executions of a spec file are considered for counting the suite size

Customization

Only test recordings matching the filters will be included for metric calculation.

• Date Range - include items recorded within the specified period

• Tag - include items with the matchingPlaywright Tags

• Author - include items with the matching Git Author (seeCommit Information)

• Branch - include items with the matching Git Branch (seeCommit Information)

• Group - include items recorded for particular group (e.g. Firefox or Chromium)

• Search by spec name - narrow down the results by spec name

Controls & Settings

• Click on a column header to sort the tests by the corresponding column, click again to change the sorting order.

• Click on the Export icon to download the data in JSON format

• Click in the Settings icon to customize the view:

  • Include Failed Executions - include or exclude the failed execution in calculating the avg. duration

Spec Explorer Settings

Use Cases

• The flakiest Spec Files from the selected period for a specific branch.

• The Failure Rate trends for specific branches for the past month.

• The most long-running Spec Files and how they changed their duration over time.

• The Spec Files that experienced the most timeouts in the past weeks.

Individual Spec File Analysis

Clicking on an individual spec file will reveal a dedicated view of its performance, including detailed execution History and performance Metrics.

History - Spec Files Explorer

Spec File Performance

Next Steps

View Reference for detailed information on test History and Performance Charts.