Spec Files Explorer
Spec-level health and performance dashboard - flakiness, failure rate, duration, suite size
Last updated
Spec-level health and performance dashboard - flakiness, failure rate, duration, suite size
Last updated
The Spec Files Explorer allows users to track and analyze spec files performance using various metrics like:
Duration
Failure Rate
Flakiness Rate
Timeout Rate
Suite Size
Currents calculates the metrics based on the executions recorded during the provided date range and filters. A typical use case is to monitor the performance of the spec files that cause the degraded performance of the overall test suite - e.g. spec files with the highest flakiness rate, longest duration etc.
The average duration of , excluding timed-out or cancelled spec files. You can exclude or include failed executions for calculating the average duration in Settings (see below).
The failure rate is the percentage of spec file executions that failed. A spec file is failed if it had 1 or more failed tests. See Spec File Status.
The percentage of spec file executions that timed out. A spec file is marked as timed out if we didn't report its test results before exceeding its run's timeout. See Run Timeouts.
The percentage of spec file executions that had 1 or more flaky tests. See Flaky Tests.
The number of tests in a spec file (including skipped tests).
How many executions were included during in the selected period and match the filters.
Only test recordings matching the filters will be included for metric calculation.
Date Range - include items recorded within the specified period
Group - include items recorded for particular group (e.g. Firefox or Chromium)
Search by spec name - narrow down the results by spec name
Click on a column header to sort the tests by the corresponding column, click again to change the sorting order.
Include Failed Executions - include or exclude the failed execution in calculating the avg. duration
The flakiest Spec Files from the selected period for a specific branch.
The Failure Rate trends for specific branches for the past month.
The most long-running Spec Files and how they changed their duration over time.
The Spec Files that experienced the most timeouts in the past weeks.
Clicking on an individual spec file will reveal a detailed drill-down of the selected spec file performance, including a detailed history of execution, top errors, etc. See Spec Files Performance.
Explore individual spec details by clicking on its name and opening Spec Files Performance view; and also schedule containing the highlights of the Spec Files health and performance metrics delivered to your inbox.
Tag - include items with the matching
Author - include items with the matching Git Author (see )
Branch - include items with the matching Git Branch (see )
Click on the Export icon to download the data in JSON format
Click in the Settings icon to customize the view:
