Reporting Strategy

This guide aims to help readers understand how to structure and organize the test result reporting in Currents. It covers key concepts such as project, runs and groups hierarchy, test result handling, and integration strategies.

By following this guide, users can optimize their test reporting workflow and gain clearer insights into their test runs.

Currents Reporting Terms

Project

The top-level entity representing reporting destination. Each organization can have multiple projects, there’s no limit on the number of projects.

You specify the project when running tests in your CI pipeline using environment variables CURRENTS_PROJECT_ID or via reporter configuration. Please refer to reporter documentation for available options.

Test Results and History

• Test results from different projects remain separate, even for identical tests.

• Test history displays results only from the same project.

• Performance metrics only include test recordings from the same project.

Integrations

Each project has its own integrations with 3rd parties (see Integrations). However, multiple projects can be connected to the same destination.

For example, enabling a GitHub integration allows selecting the same GitHub organization and repository for different projects. However, this may result in duplicate PR comments originating from separate projects connected to the same repository.

Run

A recording of test results from a CI run, uniquely identified by a CI Build ID, git commit information and CI provider execution details.

By default each run represents a build (or execution of a CI pipeline), however, you can send results from different stages of a pipeline or even different pipelines to the same run using the same CI Build ID value. See CI Build ID.

Group

A collection of spec files and the corresponding test results. For example:

• in Playwright you can create two Playwright projects that run the same tests for chrome and firefox - Currents will create a separate group, accordingly.

• in Postman each “collection” corresponds a separate group.

Each group can contain multiple spec files and test results.

Spec File

A file containing one or more test cases, identified by its reporter filesystem path. Currents collects certain performance metrics for specs. Spec file can contain zero or more test recordings.

Test Recording

A single test execution result, including attempts, attachments (screenshots, videos, traces, arbitrary attachments, visual diff results, annotations, error details. Each test record can have multiple attempts.

Attempt

An attempt to run of a test case. Multiple attempts may occur depending on retry strategies.

Reporting Scenarios

You can implement various reporting scenarios using a combination of Projects, Runs and Groups. Consider the following popular scenarios.

Single product / repository

You have a single product (and a repository) and a testing suite that runs as part of a CI pipeline, all tests run together, optionally multiple groups (e.g. chromium and firefox ) will be automatically detected.

It is recommended to have a single project and create a unique run on every invocation of the CI pipeline.

• Create a single project

• Create a unique run for each CI invocation

• Create single or multiple groups within each run

Multiple standalone products / repositories

For bigger organizations, each product has its own repository and testing suite that runs as part of a CI pipeline.

• Create a dedicated project for each product / repository

• Create a single run for each CI invocation

• Create a single or multiple groups for each run

Mixed reporting, multiple environments

A more complicated scenario is when you have a testing suite that runs in different "environments" and you need to decide how to organize the test results. For example:

• running different Playwright projects in separate CI steps

• running the same set of tests in multiple environments, e.g. based on locale (en-us, en-ca ) or a domain (com, .co.uk)

• running a subset of tests based on tags or glob pattern, e.g. playwright test --grep @desktop

• running a subset of tests from different CI steps (or different pipelines) while separating the results

To implement the variety of possible scenarios you need to use the combination of Project, CI build ID, Groups and Tags, become familiar with the limitations and implications.

Example

For example, consider testing an e-commerce web app with a slightly different set of Playwright tests for different domains (.com, .co.uk etc.).

You've defined the following Playwright projects

export default {
  // ...
  projects: [
    {
      name:"UK",
      testMatch: ["./e2e/shared/*.spec.ts", "./e2e/uk/*.spec"],
      use: {
        baseURL: "example.co.uk" // 🇬🇧
      }
    },
    {
      name: "US",
      testMatch: ["./e2e/shared/*.spec.ts", "./e2e/us/*.spec"],
      use: {
        baseURL: "example.com" // 🇺🇸
      }
    },
  ]
}

Let's consider various setups and the implications.

Single project, single run, multiple groups

Single project, multiple runs, single group

Using Tags

Regardless of the reporting strategy we recommend to annotate your tests and executions with tags. Adding tags allows granular access to the data.

• Read more about using Playwright Tags to dynamically add tags to runs, groups and tests

• Consider using removeTitleTagin @currents/playwright to remove tags from test title and keep test history consistent