Skip to content

annotations

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

annotations

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
src
src
 
 
CHANGELOG.md
CHANGELOG.md
 
 
README.md
README.md
 
 
package.json
package.json
 
 
tsconfig.json
tsconfig.json
 
 

README.md

@lytics/playwright-annotations

Generic annotation framework for Playwright tests + Contentstack/Lytics conventions

npm version License: MIT

What This Package Provides

For Everyone 🌍

A generic annotation framework that any team can use:

  • ✅ Type-safe annotation helpers
  • ✅ Flexible validation framework
  • ✅ Works with any annotation schema
  • ✅ ESLint plugin (coming soon)

For Contentstack/Lytics Teams 🏢

Pre-built conventions for journey-driven testing:

  • ✅ testSuiteName / journeyId / testCaseId pattern
  • ✅ Validation rules for naming hierarchy
  • ✅ Convenience helpers

You can use our pattern or define your own!

Installation

npm install @lytics/playwright-annotations
# or
pnpm add @lytics/playwright-annotations
# or
yarn add @lytics/playwright-annotations

Quick Start

Option A: Use Contentstack Conventions (Recommended)

import { test as base } from "@playwright/test";
import { pushSuiteAnnotation, pushTestAnnotations } from "@lytics/playwright-annotations";

test.describe("Account Security - View Tokens @smoke", () => {
  test.beforeEach(async ({}, testInfo) => {
    pushSuiteAnnotation(testInfo, "ACCOUNT_SECURITY");
  });

  test("allows user to view access tokens", async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
      testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
    });

    // Your test implementation...
  });
});

Option B: Define Your Own Schema

import { pushAnnotations, createValidator } from "@lytics/playwright-annotations";
import type { TestAnnotations } from "@lytics/playwright-annotations";

// 1. Define your annotation schema
interface MyTeamAnnotations extends TestAnnotations {
  featureArea: string;
  ticketId: string;
  priority: 'high' | 'medium' | 'low';
}

// 2. Create a validator for your schema
const validateMyAnnotations = createValidator<MyTeamAnnotations>({
  rules: [
    (annotations) => ({
      valid: Boolean(annotations.featureArea),
      errors: annotations.featureArea ? [] : ['featureArea is required'],
      warnings: [],
    }),
    (annotations) => ({
      valid: /^[A-Z]+-\d+$/.test(annotations.ticketId),
      errors: /^[A-Z]+-\d+$/.test(annotations.ticketId) 
        ? [] 
        : ['ticketId must match pattern: PROJECT-123'],
      warnings: [],
    }),
  ]
});

// 3. Use in your tests
test("my feature", async ({}, testInfo) => {
  const annotations: MyTeamAnnotations = {
    featureArea: 'auth',
    ticketId: 'JIRA-123',
    priority: 'high'
  };

  const validation = validateMyAnnotations(annotations);
  if (!validation.valid) {
    throw new Error(`Invalid annotations: ${validation.errors.join(', ')}`);
  }

  pushAnnotations(testInfo, annotations);

  // Your test implementation...
});

Contentstack Conventions (Reference Implementation)

The Three Required Annotations

Annotation Scope Purpose Example
testSuiteName Suite (1:N journeys) Organizational grouping "ACCOUNT_SECURITY"
journeyId Journey (1:N test cases) Unique user flow "ACCOUNT_SECURITY_VIEW-TOKENS"
testCaseId Test Case (unique) Specific scenario "ACCOUNT_SECURITY_VIEW-TOKENS_VALID"

Hierarchy

ACCOUNT_SECURITY (testSuiteName)
  ├── ACCOUNT_SECURITY_VIEW-TOKENS (journeyId)
  │   ├── ACCOUNT_SECURITY_VIEW-TOKENS_VALID (testCaseId)
  │   └── ACCOUNT_SECURITY_VIEW-TOKENS_EMPTY (testCaseId)
  └── ACCOUNT_SECURITY_CREATE-TOKEN (journeyId)
      └── ACCOUNT_SECURITY_CREATE-TOKEN_VALID (testCaseId)

Validation Rules

  1. ✅ Required: All three annotations must be present
  2. ✅ Hierarchy: testCaseId must start with journeyId (error)
  3. ✅ Uniqueness: testCaseId cannot equal journeyId (must add suffix)
  4. ⚠️ Recommendation: journeyId should start with testSuiteName (warning)

Flexible Delimiters

Hyphens and underscores are normalized for comparison:

// All valid:
testSuiteName: "HOME_DASHBOARD"
journeyId: "HOME-DASHBOARD_RECIPE"  // Hyphen variant OK
testCaseId: "HOME-DASHBOARD_RECIPE_VALID"

API Reference

Generic API

Types

interface TestAnnotations {
  [key: string]: string | string[] | undefined;
}

Functions

pushAnnotations<T>(testInfo, annotations)

Push any annotations to a test (generic).

pushAnnotations(testInfo, {
  myField: 'value',
  myOtherField: 'another value'
});

getAnnotations<T>(test)

Extract annotations from a test case (generic).

const annotations = getAnnotations<MyAnnotations>(test);

createValidator<T>(config)

Factory for creating custom validators.

const validator = createValidator<MyAnnotations>({
  rules: [
    (annotations) => ({
      valid: Boolean(annotations.requiredField),
      errors: annotations.requiredField ? [] : ['requiredField is required'],
      warnings: [],
    }),
  ]
});

Contentstack-Specific API

Types

interface ContentstackAnnotations {
  testSuiteName: string;
  journeyId: string;
  testCaseId: string;
  tags?: string[];
}

Functions

pushSuiteAnnotation(testInfo, testSuiteName)

Set suite annotation (use in beforeEach).

test.beforeEach(async ({}, testInfo) => {
  pushSuiteAnnotation(testInfo, "ACCOUNT_SECURITY");
});

pushTestAnnotations(testInfo, { journeyId, testCaseId })

Set test annotations with validation.

test("my test", async ({}, testInfo) => {
  pushTestAnnotations(testInfo, {
    journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
    testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
  });
});

pushContentstackAnnotations(testInfo, annotations)

Set all annotations at once.

pushContentstackAnnotations(testInfo, {
  testSuiteName: "ACCOUNT_SECURITY",
  journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
  testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
});

getContentstackAnnotations(test)

Extract Contentstack annotations from a test.

const annotations = getContentstackAnnotations(test);
// Returns: { testSuiteName, journeyId, testCaseId } | null

validateContentstackAnnotations(annotations)

Validate Contentstack annotation format.

const result = validateContentstackAnnotations({
  testSuiteName: "ACCOUNT_SECURITY",
  journeyId: "ACCOUNT_SECURITY_VIEW-TOKENS",
  testCaseId: "ACCOUNT_SECURITY_VIEW-TOKENS_VALID",
});

if (!result.valid) {
  console.error(result.errors);
}

Why Annotations?

Annotations enable:

  • 📊 Observability - Centralized dashboards showing test health
  • 🔗 Traceability - Link tests to user journeys and requirements
  • 📈 Trend Analysis - Track test stability over time
  • 🎯 Coverage Tracking - Identify gaps in test coverage
  • 🔍 Querying - Find all tests for a journey or suite

Integration with Other Packages

With Reporter

Annotations are consumed by @lytics/playwright-reporter. Create a reporter file:

// reporter.ts
import { CoreReporter } from '@lytics/playwright-reporter';
import { FirestoreAdapter } from '@lytics/playwright-adapters/firestore';

class CustomReporter extends CoreReporter {
  constructor() {
    super({
      adapters: [
        new FirestoreAdapter({
          projectId: 'my-project',
          collections: { /* ... */ },
          // Adapter uses annotations for querying and grouping
        }),
      ],
    });
  }
}

export default CustomReporter;

Reference it in your config:

// playwright.config.ts
export default {
  reporter: [['list'], ['./reporter.ts']]
};

With Journey Tools

Journey metadata drives test generation:

import { TestGenerator } from '@lytics/playwright-journey';

const generator = new TestGenerator({
  // Generates tests with proper annotations from journey metadata
});

Examples

Example 1: Basic Contentstack Usage

import { test } from "@playwright/test";
import { pushSuiteAnnotation, pushTestAnnotations } from "@lytics/playwright-annotations";

test.describe("User Management @smoke", () => {
  test.beforeEach(async ({}, testInfo) => {
    pushSuiteAnnotation(testInfo, "USER-MANAGEMENT");
  });

  test("admin can create user", async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: "USER-MANAGEMENT_CREATE-USER",
      testCaseId: "USER-MANAGEMENT_CREATE-USER_VALID",
    });

    // Test implementation...
  });

  test("validates required fields", async ({}, testInfo) => {
    pushTestAnnotations(testInfo, {
      journeyId: "USER-MANAGEMENT_CREATE-USER",
      testCaseId: "USER-MANAGEMENT_CREATE-USER_INVALID",
    });

    // Test implementation...
  });
});

Example 2: Custom Schema

import { pushAnnotations, createValidator } from "@lytics/playwright-annotations";

interface MyAnnotations {
  epic: string;
  story: string;
  severity: 'blocker' | 'critical' | 'major' | 'minor';
}

const validateMyAnnotations = createValidator<MyAnnotations>({
  rules: [
    (a) => ({
      valid: Boolean(a.epic && a.story),
      errors: !a.epic || !a.story ? ['epic and story are required'] : [],
      warnings: [],
    }),
  ]
});

test("my test", async ({}, testInfo) => {
  const annotations: MyAnnotations = {
    epic: 'AUTH-2023',
    story: 'AUTH-2023-01',
    severity: 'critical'
  };

  const result = validateMyAnnotations(annotations);
  if (!result.valid) {
    throw new Error(result.errors.join(', '));
  }

  pushAnnotations(testInfo, annotations);
});

Contributing

See CONTRIBUTING.md in the repository root.

License

MIT

Related Packages