Insights and Analytics
Test and Run Insights and Analytics Guide
Last updated
Was this helpful?
Test and Run Insights and Analytics Guide
Last updated
Was this helpful?
Use Insights to explore the aggregated metrics related to your test suite performance:
changes and trends in duration and success rate of your builds and tests over time,
test suite flakiness and stability.
Customize the reports by applying various filters for a more focused and meaningful insights.
All charts support filtering by tag, git author, and git branch. In addition:
Suite Size, Test Results, and Test Flakiness charts also support the group filter.
Test Results and Test Flakiness charts also support the Playwright Annotationsfilter.
The aggregated metrics are based in the samples collected during the selected Date Range.
Represents the distribution of the outcomes of your builds/runs over time.
Overall runs: overall runs created during the selected period, regardless of their completion or the end state.
Failed runs: runs that have one or more failed tests. Failed runs count includes cancelled and timed-out tests.
Passed runs: runs that have zero failed tests.
Pass Rate: passed runs / overall runs.
Represents the daily/weekly average duration of fully reported runs. A fully reported run is a run that wasn't cancelled and didn't time out.
Please note that cancelled or timed-out runs are excluded from the report.
Represents the distribution of runs by their completion over time.
Overall runs: overall runs created during the selected period, regardless of their completion or the end state.
Fully reported runs: runs that weren't cancelled and didn't time out.
Cancelled runs: see Cancelling Runs
Timed out runs: see Run Timeouts
Represents the maximum daily/weekly amount of specs/tests for fully completed runs. A fully reported run is a run that wasn't cancelled and didn't time out.
Spec files: the maximum number of spec files detected in a run for the selected period
Tests: the maximum number of tests detected in a run for the selected period
Please note that cancelled or timed-out runs are excluded from the report.
Shows the distribution of test outcomes over time.
Total tests: overall tests recorded for the selected period, regardless of their outcome.
Passed tests: tests that were successfully completed on all attempts without any exceptions or errors during their execution.
Failed tests: tests that either failed or were skipped because of an error in beforeEach/beforeAll
Ignored test: tests that weren't run, e.g. it.skip()
Success Rate: passed / (total - pending)
Shows the distribution of flaky tests over time.
Flaky tests: overall flaky tests detected for the period. See Flaky Tests
Flakiness rate: flaky tests count / passed tests count
