fix(runner): stop reporting context-window size as a run's token usage #6857
Annotations
10 errors, 3 warnings, and 1 notice
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
6) ../oss/tests/playwright/acceptance/playground/index.ts:141:23 › Playground: Run Variant › Should run single view variant for chat @scope:playground @coverage:smoke @coverage:light @coverage:full @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow › And the active project has a configured test provider
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/playground/index.ts:159:17
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/playground/index.ts:158:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
5) ../oss/tests/playwright/acceptance/observability/index.ts:327:9 › Observability: view traces › should switch between trace tabs and see filtered rows @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:332:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
5) ../oss/tests/playwright/acceptance/observability/index.ts:327:9 › Observability: view traces › should switch between trace tabs and see filtered rows @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:332:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
4) ../oss/tests/playwright/acceptance/observability/index.ts:290:13 › Observability: view traces › Should open a span and drill into its attributes (retry-eligible) › should open a span and drill into its attributes @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:295:17
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
4) ../oss/tests/playwright/acceptance/observability/index.ts:290:13 › Observability: view traces › Should open a span and drill into its attributes (retry-eligible) › should open a span and drill into its attributes @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:295:17
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
3) ../oss/tests/playwright/acceptance/observability/index.ts:244:9 › Observability: view traces › should filter traces by span name or attribute @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:249:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
3) ../oss/tests/playwright/acceptance/observability/index.ts:244:9 › Observability: view traces › should filter traces by span name or attribute @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:249:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
2) ../oss/tests/playwright/acceptance/observability/index.ts:206:9 › Observability: view traces › should filter traces by date range and by app @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:211:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts#L199
2) ../oss/tests/playwright/acceptance/observability/index.ts:206:9 › Observability: view traces › should filter traces by date range and by app @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toBeVisible() failed
Locator: getByRole('heading', { name: 'LLMs' })
Expected: visible
Timeout: 15000ms
Error: element(s) not found
Call log:
- Expect "toBeVisible" with timeout 15000ms
- waiting for getByRole('heading', { name: 'LLMs' })
at ../../../../tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199
197 |
198 | await uiHelpers.expectPath("/settings")
> 199 | await expect(page.getByRole("heading", {name: "LLMs"})).toBeVisible({
| ^
200 | timeout: 15000,
201 | })
202 | await expect(getCustomProvidersSection(page)).toBeVisible({timeout: 15000})
at navigateToModels (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:199:61)
at Object.ensureMockProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:368:5)
at Object.ensureTestProvider (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/providerHelpers/index.ts:508:17)
at runPlaygroundAndGoToObservability (/home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:74:5)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/observability/index.ts:211:13
|
|
Run web tests:
web/tests/tests/fixtures/base.fixture/apiHelpers/index.ts#L337
1) ../oss/tests/playwright/acceptance/app/app-management.ts:142:13 › App Management › should render the app overview page with environment cards and variant list @scope:apps @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:fast › And a fresh completion app exists in the project
Error: page.waitForResponse: Test ended.
at ../../../../tests/tests/fixtures/base.fixture/apiHelpers/index.ts:337
335 |
336 | // 1. POST /workflows/ — create the workflow
> 337 | const createWorkflowPromise = page.waitForResponse((response) => {
| ^
338 | if (
339 | !response.url().includes("/workflows") ||
340 | response.url().includes("/query") ||
at createApp (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/apiHelpers/index.ts:337:40)
at Object.createApp (/home/runner/work/agenta/agenta/web/tests/tests/fixtures/base.fixture/apiHelpers/index.ts:736:24)
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/app/app-management.ts:163:29
at /home/runner/work/agenta/agenta/web/oss/tests/playwright/acceptance/app/app-management.ts:155:13
|
|
Complete job
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/cache@v4, actions/setup-node@v4, actions/upload-artifact@v4, pnpm/action-setup@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Upload test report
No files were found with the provided path: web/tests/reports/oss/
web/packages/*/test-results/. No artifacts will be uploaded.
|
|
Publish web test results
Could not find any files for web/tests/results/oss/junit.xml, web/packages/*/test-results/*junit.xml
|
|
Run web tests
9 failed
../oss/tests/playwright/acceptance/observability/index.ts:206:9 › Observability: view traces › should filter traces by date range and by app @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/observability/index.ts:244:9 › Observability: view traces › should filter traces by span name or attribute @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/observability/index.ts:290:13 › Observability: view traces › Should open a span and drill into its attributes (retry-eligible) › should open a span and drill into its attributes @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/observability/index.ts:327:9 › Observability: view traces › should switch between trace tabs and see filtered rows @scope:observability @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/playground/index.ts:141:23 › Playground: Run Variant › Should run single view variant for chat @scope:playground @coverage:smoke @coverage:light @coverage:full @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/playground/index.ts:292:23 › Playground: Run Variant › should load the correct variant when opened via a deep link with a revisions param @scope:playground @coverage:smoke @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:fast
../oss/tests/playwright/acceptance/prompt-registry/index.ts:135:9 › Prompt Registry: workflow revisions › should open prompt details from prompt registry @scope:playground @coverage:smoke @coverage:light @coverage:full @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:fast
../oss/tests/playwright/acceptance/settings/model-hub.ts:78:9 › Settings: Model Hub › should allow full add provider @scope:settings @coverage:smoke @coverage:light @coverage:full @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:fast
../oss/tests/playwright/acceptance/settings/model-hub.ts:99:9 › Settings: Model Hub › should configure a standard provider key and verify it is listed @scope:settings @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:fast
5 flaky
../oss/tests/playwright/acceptance/app/app-management.ts:142:13 › App Management › should render the app overview page with environment cards and variant list @scope:apps @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:fast
../oss/tests/playwright/acceptance/playground/index.ts:178:23 › Playground: Run Variant › Should update the prompt and save the changes @scope:playground @coverage:smoke @coverage:light @coverage:full @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/playground/index.ts:219:23 › Playground: Run Variant › should save the current changes as a new variant @scope:playground @coverage:smoke @coverage:light @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/playground/index.ts:251:23 › Playground: Run Variant › should open compare mode and display two variants side by side @scope:playground @coverage:light @coverage:full @path:happy @lens:functional @cost:free @license:oss @ROLE:owner @case:typical @speed:slow
../oss/tests/playwright/acceptance/use-api/index.ts:183:9 › Registry: use API snippets › should show variant TypeScript snippet for Fetch Prompt/Config and Invoke LLM @scope:deployment @covera
|
background
wait
wait-all
cancel
parallel
Loading