Playwright Annotations

Using Playwright Annotations to enhance reporting to Currents dashboard

Requires @currents/playwright 1.5.0+

Playwright Annotations is a flexible way to add additional information about tests, like:

  • ownership information

  • metadata

  • links to external resources (Jira ticket, GitHub issue)

  • notes

Together with Playwright Tags it allows augmenting your testing suite with metadata for easier managing, better reporting and improved integrations.

You can add an annotation to a test by:

  • adding annotations object to test definition or

  • calling testInfo.annotations.push

For example:

test("annotated test", {
    annotation: {
      type: "issue",
      description: "https://github.com/microsoft/playwright/issues/23180",
    },
}, async ({ page }, testInfo) => {
  testInfo.annotations.push({
    type: "note",
    description: "This is a note",
  });

  testInfo.annotations.push({
    type: "jira",
    description: "https://jira.company.io/ticket/JIRA-123",
  });

  testInfo.annotations.push({
    type: "owner",
    description: "johnsmith",
  });
});

Currents shows the annotations for the test:

Playwright annotations in Currents

Limitations

Currents applies the following rules when parsing annotations:

  • types: skip, fixme, fail, slow are reserved by Playwright

  • 32 max distinct annotations per test, extra annotations will be removed (sorted by the order of appearance)

  • type field is limited to 256 characters, the values are trimmed and truncated to the max length

  • description field is limited to 2048 characters, the values are trimmed and truncated to the max length

  • If type is empty after trimming, the annotation is ignored

Source and Deduplication

Annotations can originate from test case definition or at runtime from test execution attempt.

  • Currents deduplicates annotations with exactly the same type, description and source.

  • Currents removes attempt-level annotation if there's an equivalent test-case annotation

Annotation: Test Owner

While Currents displays all the annotations related to a test, some annotation have a special meaning, for example - test owner.

To designate an owner of a test, add annotation with type: owner, for example:

testInfo.annotations.push({
  type: "owner",
  description: "johnsmith",
});

The value will appear in various areas of the dashboard so that your team can quickly identify the who owns the test.

Showing test owner using annotations in Currents

Annotation: Slack Notifications

Annotation of type notify:slack activates Slack mentions for failed tests - when Currents detects a failed test with notify:slack annotation, it will trigger Slack notification according to the following convention:

  • type: "notify:slack", description: "user:userId" - will notify user with the corresponding userId, userId can be either a Slack UserId or Slack Username

  • type: "notify:slack", description: "team:teamId" - will notify team with the corresponding teamId, (see how to retrieve Slack Team id)

You can combine the values to activate multiple notifications, for example:

{
  "type": "notify:slack",
  "description": "user:U01RWNBFGER, team:S07JCUP81EG"
}

For example, the following annotations will define test owner and activate slack notifications:

test(
    "my failed test",
    {
      annotation: [
        {
          type: "owner", // This shows the user in the dashboard
          description: "Miguel Langarano",
        },
        {
          type: "notify:slack", // This notifies the user in Slack
          description: "user:U01RWNBFGER",
        },
        {
          type: "notify:slack", // This notifies a group in Slack
          description: "team:T01S60385HA",
        },
      ],
    },
    async ({ page }) => {
      expect(true).toBe(false);
    }
);
Test owner shown in Currents dashboard
Slack notification to specific user and group in Slack

Annotation: Custom Metrics

Custom metric is an experimental feature. We are gathering feedback to refine and improve it.

Recording Custom Metrics

Annotation of type currents:metric allows tracking arbitrary metrics associated with your tests, for example:

  • page performance

  • accessibility score

  • memory consumption

  • network response timing

Use annotation type of currents:metric and a serialized JSON object in description to define a metric. For example:

{
  type: "currents:metric",
  description: JSON.stringify({
    "name": "memory_usage", 
    "value": 540.3, 
    "type": "float",
    "unit": "mb"
  }),
}

The JSON string must contain name and value. And optionally can contain type and unit .

Parameter
Type
Description

name*

string

The name of the metric you want to track.

value*

number

The current value of the metric you want to track.

type

enum

The value type. Values: float, integer. Default: float

unit

enum

The unit to show in the dashboard. Values: none, ms, s, %, b , kb , mb , gb Default: none

For example, the following test captures page load time as a custom metric metrics:

test("performance test example", async ({ page }) => {
  await page.goto(`https://example.com`);
  const [performanceTiming] = await page.evaluate(() => {
    const [timing] = performance.getEntriesByType(
      "navigation"
    ) as PerformanceNavigationTiming[];
    return [timing];
  });
  // Get the start to load event end time
  const startToLoadEventEnd =
    performanceTiming.loadEventEnd - performanceTiming.startTime;
    
  // Add the custom metric to the annotations
  test
    .info()
    .annotations.push({
      type: "currents:metric",
      description: JSON.stringify({
        name: "page-performance",
        value: startToLoadEventEnd,
        type: "float",
        unit: "ms",
      }),
    });
});

Browsing Custom Metrics

The custom metrics are avaialble in Test Results chart. Use "Custom Metric" control to browse the available metrics and aggregation functions.

Custom metric selection

Last updated

Was this helpful?