Skip to content

feat: Frontend test optimization - smart selection, build caching, and flaky test fixes#9294

Open
bmartel wants to merge 1 commit intodevelopfrom
fix-frontend-test-runner
Open

feat: Frontend test optimization - smart selection, build caching, and flaky test fixes#9294
bmartel wants to merge 1 commit intodevelopfrom
fix-frontend-test-runner

Conversation

@bmartel
Copy link
Contributor

@bmartel bmartel commented Jan 29, 2026

Summary

This PR significantly improves frontend test efficiency, stability, and cost-effectiveness through:

  • Smart test selection: Skip tests for non-code changes (CSS, docs, configs)
  • Build optimization: Production builds with caching instead of dev server
  • Parallel execution: Use both CPU cores with cypress-parallel
  • Flaky test fixes: Replace arbitrary waits with deterministic state checks
  • Test quarantine: Isolate known flaky tests to unblock PRs

Changes

1. Smart Test Selection (select-frontend-tests.sh)

Three-tier detection system that analyzes changed files:

Tier Action Examples
Skip No tests *.scss, *.css, *.md, *.yml, assets
All Tests Full suite core/**, stores/**, package.json
Targeted Specific tests Audio changes → audio tests only
Fallback Smoke tests Unknown code changes

2. Build & Serve Optimization

Before: Dev server with HMR, compiles on-demand, ~30-60s startup
After: Production build + static serve, ~2s startup, fully cacheable

  • Added ci Babel environment for coverage-instrumented production builds
  • Added LSF_TEST_MODE flag to load feature flags in production mode
  • Build cache invalidates on source, deps, or config changes

3. Parallel Test Execution

Enabled cypress-parallel to use both CPU cores on ubuntu-latest runners:

npx cypress-parallel -t 2 -d "libs/editor/tests/integration/e2e"

4. Flaky Test Fixes

Replaced non-deterministic waits with proper state synchronization:

// Before (flaky)
cy.wait(1000);
VideoView.captureCanvas("canvas");

// After (deterministic)
VideoView.waitForRegionInKonvaByIndex(0);
VideoView.waitForStableState();
VideoView.captureCanvas("canvas");

Fixed tests:

  • video/regions.cy.ts
  • video/timeline_region_loop.cy.ts
  • audio/audio_regions.cy.ts

5. Flaky Test Quarantine

Created tests-yarn-integration-flaky.yml that runs quarantined tests with continue-on-error: true - visible but non-blocking while fixes are validated.

Expected Impact

Metric Before After
CSS/SCSS changes 70 min 0 min (skipped)
Docs/config changes 70 min 0 min (skipped)
Full test suite ~35 min ~20 min (parallel)
Retry rate 30-50% <10%
Build time (cached) 30-60s ~2s

Test Plan

  • Verify test selection script correctly categorizes file changes
  • Confirm build cache hits/misses work correctly
  • Validate flaky tests pass consistently with new waits
  • Check coverage still uploads correctly with production builds

…iles changed with quarantine for flaky tests
@bmartel bmartel requested review from a team, hlomzik and nick-skriabin as code owners January 29, 2026 18:41
@netlify
Copy link

netlify bot commented Jan 29, 2026

Deploy Preview for label-studio-docs-new-theme canceled.

Name Link
🔨 Latest commit 6322ad2
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-docs-new-theme/deploys/697ba9d37898b90008383245

@netlify
Copy link

netlify bot commented Jan 29, 2026

Deploy Preview for heartex-docs canceled.

Name Link
🔨 Latest commit 6322ad2
🔍 Latest deploy log https://app.netlify.com/projects/heartex-docs/deploys/697ba9d3d0441f0009483254

@netlify
Copy link

netlify bot commented Jan 29, 2026

Deploy Preview for label-studio-playground ready!

Name Link
🔨 Latest commit 6322ad2
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-playground/deploys/697ba9d303d70c00083b2361
😎 Deploy Preview https://deploy-preview-9294--label-studio-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions bot added the ci label Jan 29, 2026
@netlify
Copy link

netlify bot commented Jan 29, 2026

Deploy Preview for label-studio-storybook ready!

Name Link
🔨 Latest commit 6322ad2
🔍 Latest deploy log https://app.netlify.com/projects/label-studio-storybook/deploys/697ba9d31539b80007011db4
😎 Deploy Preview https://deploy-preview-9294--label-studio-storybook.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@bmartel bmartel changed the title ci: Allow optimized selective frontend test running based on actual files changed with quarantine for flaky tests feat: Frontend test optimization - smart selection, build caching, and flaky test fixes Jan 29, 2026
@github-actions github-actions bot added the feat label Jan 29, 2026
@robot-ci-heartex
Copy link
Collaborator

This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants