chore: mcp for qa workflows (#28)

* chore: mcp for qa workflows

* chore: add semantic objects

* chore: add test-website, minor WebDriver protocol fixes, add express server

* chore: bump to node 24

* feat: example using semantic objects

* chore: simplified semantic objects authoring, support jsx snapshots

* feat: comprehensive TypeScript strict typing and Node.js 22+ modernization

- Complete elimination of :any types throughout codebase
- Replace all any with unknown and proper interfaces for strict typing
- Modernize to Node.js 22+ native AbortSignal.any() without conditionals
- Implement ECMAScript 2024 Explicit Resource Management with Symbol.asyncDispose
- Standardize Vitest to ^3.2.0 across all packages for consistency
- Fix semantic discovery script generation with module import cleaning
- Resolve port conflicts with dedicated environments (3000/5032 vs 3002/5033)
- Remove emojis and maintain professional code standards
- Migrate jest-web to vitest-web for framework consistency
- Add comprehensive TypeScript declaration files
- Ensure all tests pass with clean execution

All objectives achieved: strict typing, modern Node.js features, robust infrastructure

* feat: migrate from Lerna to Nx monorepo management

- Install Nx 22.0.1 alongside existing Lerna setup
- Add comprehensive nx.json configuration with intelligent caching
- Configure targetDefaults for build, test, and prepare tasks
- Set up proper cache inputs/outputs for TypeScript compilation
- Replace lerna run commands with nx run-many for better performance
- Add affected change detection commands (test:affected, build:affected)
- Include dependency graph visualization and workspace utilities
- Remove lerna.json as Nx now handles monorepo orchestration
- Maintain existing publishing workflow through lerna commands
- Enable intelligent build caching and parallel execution

Benefits achieved:
- Faster builds with intelligent caching (cache hits already observed)
- Better dependency tracking and affected change detection
- Parallel task execution with optimal scheduling
- Visual dependency graph and workspace insights
- Maintained compatibility with existing workflows

* feat: complete migration from Lerna to Nx with semantic versioning

- Remove Lerna dependency and configuration completely
- Configure Nx release with conventional commits and independent versioning
- Add @nx/js plugin for TypeScript/JavaScript release functionality
- Update GitHub workflows to use Nx release instead of Lerna publish
- Configure proper npm authentication and registry settings
- Add intelligent caching and affected change detection to CI/CD
- Enable automatic changelog generation and git tagging
- Maintain semantic versioning with conventional commit standards

Migration benefits:
- Unified toolchain: build, test, and release all through Nx
- Better caching and parallelization in CI/CD pipelines
- Intelligent affected change detection for faster builds
- Automatic changelog generation with conventional commits
- Independent package versioning with proper git tagging
- Streamlined publishing workflow with semantic release

* feat: reorganizing the mcp

- Add configurable ChromeDriver port support (default: 9515)
- Restructure monolithic roadkill-mcp.ts into modular mcp/ architecture
- Add VS Code MCP configuration documentation
- Clean up Unicode arrows for better cross-platform compatibility
- Modernize TypeScript strict typing and Node.js 22+ features
- Implement proper ECMAScript 2024 resource management patterns

* chore: add reexporting index.ts

* Refactor MCP tools and improve test reliability

- Move framework tools to mcp/reference.ts with reference.api and reference.example
- Remove all semantic tools while keeping file structure
- Remove step wrapper functions from all test files for cleaner async code
- Add timeout methods to Session and WebDriverClient with signal support
- Add comprehensive PageLoadStrategy documentation with usage guidance
- Fix w3schools test hanging with PageLoadStrategy.none and click timeouts
- Remove utils module and fix related imports
- Remove test script from main package (no test files remaining)
- All tests now pass reliably without hanging issues

* Remove flaky external test examples

- Delete selenium-interop and vitest-web examples (w3schools tests)
- Update workspace configuration to only include stable test-website
- Clean up package-lock.json dependencies
- Focus on reliable local test examples only

* feat: enhance DOM snapshot with HTML format and self-closing tags

- Add HTML format output option for cleaner readability
- Implement self-closing tag optimization for empty elements
- Remove depth/children/text length limits for complete DOM capture
- Maintain comprehensive whitespace normalization
- Preserve all existing optimizations (no double newlines, clean formatting)

* chore: clean up test dom

* chore: keep dom-test-selector and dom-snapshot

* chore: Rremove .js extensions and don't mention semantic obects

* chore: Add MCP utilities for dom snapshot and selector exploration

* chore: build all

* chore: run all tests always, avoid nx cache

* chore: build & test

* chore: enable ci linux ui

---------

Co-authored-by: panayot-cankov <[email protected]>

Author: panayot-cankov

.github/workflows/CD.yml

@@ -14,9 +14,6 @@ jobs:
       packages: read
 
     steps:
-      - uses: actions/setup-node@v3
-        with:
-          node-version: ^18.17
       - name: import secrets
         id: import-secrets
         uses: LanceMcCarthy/akeyless-action@v3
@@ -28,22 +25,70 @@ jobs:
               "/WebComponents/prod/tokens/PROGRESS_NPM_REGISTRY_TOKEN": "NPM_TOKEN"
             }
           export-secrets-to-environment: false
+      
       - name: git config
         run: |
           git config --global user.email "[email protected]"
           git config --global user.name "kendo-bot"
+      
       - name: git checkout
         uses: actions/checkout@v4
         with:
           token: ${{ steps.import-secrets.outputs.GH_TOKEN }}
+          fetch-depth: 0  # Needed for Nx affected commands and release
+      
+      - uses: actions/setup-node@v3
+        with:
+          node-version: ^24.1
+          cache: 'npm'
+      
+      # Setup Nx cache for faster builds
+      - name: Setup Nx cache
+        uses: actions/cache@v3
+        with:
+          path: |
+            node_modules/.cache/nx
+            .nx/cache
+          key: ${{ runner.os }}-nx-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-nx-
+      
       - name: install virtual display
-        run: sudo apt-get install xvfb
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y xvfb fluxbox x11-utils
+      
+      - name: setup virtual display
+        run: |
+          # Start Xvfb with full HD resolution
+          export DISPLAY=:99
+          Xvfb :99 -screen 0 1920x1080x24 -ac +extension GLX +render -noreset &
+          # Wait for display to be ready
+          sleep 3
+          # Start a window manager for better compatibility
+          fluxbox -display :99 &
+          # Verify display is working
+          xdpyinfo -display :99 | grep dimensions
+        env:
+          DISPLAY: :99
+      
       - name: npm install
         run: npm ci
-      - name: npm test
-        run: xvfb-run --auto-servernum npm test
-      - name: lerna publish
-        run: npm run lerna:publish
+      
+      # Build all projects to ensure everything is ready for publishing
+      - name: build all
+        run: npm run build
+      
+      # Test all projects to ensure quality before publishing
+      - name: test all
+        run: npm test
+        env:
+          DISPLAY: :99
+      
+      # Nx handles versioning and publishing with semantic versioning
+      - name: nx release
+        run: npm run release
         env:
           NPM_TOKEN: ${{ steps.import-secrets.outputs.NPM_TOKEN }}
           GH_TOKEN: ${{ steps.import-secrets.outputs.GH_TOKEN }}
+          NODE_AUTH_TOKEN: ${{ steps.import-secrets.outputs.NPM_TOKEN }}

.github/workflows/CI.yml

@@ -11,17 +11,59 @@ jobs:
 
     steps:
       - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0  # Needed for Nx affected commands
       - uses: actions/setup-node@v3
         with:
-          node-version: ^18.17
+          node-version: ^24.1
+          cache: 'npm'
+      
+      # Setup Nx cache for faster builds
+      - name: Setup Nx cache
+        uses: actions/cache@v3
+        with:
+          path: |
+            node_modules/.cache/nx
+            .nx/cache
+          key: ${{ runner.os }}-nx-${{ hashFiles('**/package-lock.json') }}
+          restore-keys: |
+            ${{ runner.os }}-nx-
+      
       - name: install virtual display
-        run: sudo apt-get install xvfb
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y xvfb fluxbox x11-utils
+      
+      - name: setup virtual display
+        run: |
+          # Start Xvfb with full HD resolution
+          export DISPLAY=:99
+          Xvfb :99 -screen 0 1920x1080x24 -ac +extension GLX +render -noreset &
+          # Wait for display to be ready
+          sleep 3
+          # Start a window manager for better compatibility
+          fluxbox -display :99 &
+          # Verify display is working
+          xdpyinfo -display :99 | grep dimensions
+        env:
+          DISPLAY: :99
+      
       - name: npm install
         run: npm ci
-      - name: npm test
-        run: xvfb-run --auto-servernum npm test
+      
+      # Build all projects
+      - name: build all
+        run: npm run build
+      
+      # Test all projects  
+      - name: test all
+        run: npm run test
+        env:
+          DISPLAY: :99
+      
       - name: npm pack
         run: npm pack --ws
+      
       - name: upload artefacts
         uses: actions/upload-artifact@v4
         with:

.gitignore

@@ -133,3 +133,10 @@ dist
 .yarn/build-state.yml
 .yarn/install-state.gz
 .pnp.*
+
+
+
+.nx/cache
+.nx/workspace-data
+.cursor/rules/nx-rules.mdc
+.github/instructions/nx.instructions.md

.vscode/settings.json

@@ -1,6 +1,12 @@
 {
     "files.exclude": {
         "**/*.js": true,
-        "**/*.js.map": true
+        "**/*.js.map": true,
+        "**/*.d.ts": true
+    },
+    "search.exclude": {
+        "**/*.js": true,
+        "**/*.js.map": true,
+        "**/*.d.ts": true
     }
 }

README.md

@@ -7,20 +7,40 @@
 
 A [node.js](https://nodejs.org/en) testing solution over the [WebDriver](https://www.w3.org/TR/webdriver2/) protocol. Will also consider [WebDriver BiDi](https://w3c.github.io/webdriver-bidi/).
 
+## Requirements
+
+- **Node.js 22+** (required for ECMAScript 2024 Explicit Resource Management)
+- **TypeScript 5.2+** (for native Disposable support)
+
 Powered by:
+ - [Vitest](https://vitest.dev) - Fast unit test framework
  - [TypeScript](https://www.typescriptlang.org)
  - [Promise based](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise)
  - [Errors with causes](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Error/cause)
  - [AbortSignals](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)
+ - [ECMAScript 2024 Explicit Resource Management](https://github.com/tc39/proposal-explicit-resource-management)
  - all the fancy tech...
 
 A [WebDriver](https://www.w3.org/TR/webdriver2/) based slim testing framework. Closes the gaps between QAs and Front-End developers by:
 
  - Sharing the same TypeScript or JavaScript language with ***Angular***, ***React*** and ***Vue*** front-end developers
  - Stay within the [nodejs](https://nodejs.org/en) ecosystem
  - Skill-transfer between QAs and Front-End devs
- - Share the lightweight VSCode IDE
- - Compile-time type-checking
+ - **ECMAScript 2024 Only** - Pure symbol-based disposables, no legacy methods
+ - **Node.js 22+ Required** - Cutting-edge JavaScript runtime features
+ - **Zero Backward Compatibility** - Clean slate approach
+
+## AI-Powered DOM Analysis
+
+Roadkill includes advanced DOM snapshot capabilities designed for AI assistants like GitHub Copilot. The `dom-snapshot` and `dom-test-selector` tools provide comprehensive page structure analysis to help craft robust selectors:
+
+- **`dom-snapshot`**: Captures complete DOM structure with element positioning, visibility states, and semantic information
+- **`dom-test-selector`**: Tests CSS/XPath selectors and shows element hierarchy with matches, parents, and children
+- **AI Integration**: Optimized for Copilot to understand page layout and suggest stable, maintainable selectors
+- **Intelligent Filtering**: Automatically excludes non-essential elements (style, script tags) for cleaner analysis
+- **Multiple Formats**: Supports both HTML and JSON output formats for different AI processing needs
+
+**Note**: Large HTML pages may trigger chat summarization in AI assistants due to content length limits. For complex pages, consider using specific root selectors to focus analysis on relevant sections.
 
 ## Demo
 To run the demo code, clone the repo, execute the ***[./packages/@progress/roadkill/example.ts](./packages/@progress/roadkill/example.ts)*** by running: