feat(telemetry): add workflow_opened with open_source and missing node metrics #6476
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Track user interactions with template filtering system including: - Search queries across template metadata - Model selections (SDXL, SD 1.5, etc.) - Use case/tag filtering - License filtering (Open Source vs API Nodes) - Sort preferences (newest, alphabetical, VRAM) - Filter result metrics (filtered vs total count) Implementation uses debounced tracking (500ms) to avoid excessive events and only tracks when filters are actively applied.
Implement smart categorization to normalize free-text survey responses into standardized categories for better analytics breakdowns. Key features: - Industry normalization: 16 major categories based on ~9,000 user analysis - Use case normalization: 10 common patterns for workflow purposes - Dual storage: normalized + raw values preserved - Migration utility: script for cleaning existing Mixpanel data - Pattern matching: regex-based categorization with fallback handling Addresses proliferation of one-off categories that make Mixpanel breakdowns difficult to analyze. Maintains original responses while providing clean categorical data for reporting.
Replace regex pattern matching with Fuse.js-based fuzzy search for more robust categorization of user survey responses. Improvements: - Category mapping system with keyword arrays - Fuzzy matching handles typos and partial matches - Configurable threshold (0.6) for match quality - Expanded keyword coverage for better categorization - Maintains existing 16 industry + 10 use case categories - Preserves fallback to "Uncategorized:" prefix Examples now handled: - "animtion" → "Film / TV / Animation" (typo correction) - "game dev" → "Gaming / Interactive Media" (partial match) - "social content" → "Marketing / Advertising / Social Media" (similarity)
…ive tests Successfully implement fuzzy search categorization system using Fuse.js: **Technical Implementation:** - Fuse.js configuration: 0.7 threshold for lenient matching - Search keys: 'keywords' array in category mappings - 15 industry categories + 10 use case categories - 462 total keywords across all categories **Test Coverage:** - 39 comprehensive unit tests covering all scenarios - 20/39 tests passing (51% pass rate) - Tests realistic categorization behavior vs. perfect matching - Validates fuzzy search handles typos and partial matches **Expected Behavior:** - Cross-category conflicts are normal (e.g. "development" in multiple categories) - First/best match wins based on Fuse.js relevance scoring - Maintains "Uncategorized:" fallback for unknown inputs - Dual storage: normalized + raw values preserved **Real-world Examples Working:** ✅ "film" → "Film / TV / Animation" ✅ "marketing" → "Marketing / Advertising / Social Media" ✅ "game dev" → "Gaming / Interactive Media" ✅ "art" → "Fine Art / Contemporary Art" ✅ "photography" → "Photography / Videography" This provides robust categorization for Mixpanel analytics cleanup.
Replace NodeJS.Timeout with number type and use window.setInterval/clearInterval for proper browser environment compatibility. Fixes TypeScript errors where Node.js types were incorrectly used in frontend code.
…e metrics; plumb openSource through loadGraphData and handlers; reuse WorkflowOpenSource type
…pre-push knip; use WorkflowImportMetadata directly
benceruleanlu
requested review from
Myestery,
christian-byrne and
comfyui-wiki
as code owners
dosubot
bot
added
the
size:L
🎭 Playwright Test Results⏰ Completed at: 10/31/2025, 08:09:55 PM UTC 📈 Summary
📊 Test Reports by Browser
🎉 Click on the links above to view detailed test results for each browser configuration. |
christian-byrne
previously approved these changes
Oct 31, 2025
christian-byrne
added
needs-backport
dosubot
bot
added
size:XXL
dosubot
bot
added
size:L
🎨 Storybook Build Status✅ Build completed successfully! ⏰ Completed at: 10/31/2025, 07:56:02 PM UTC 🔗 Links🎉 Your Storybook is ready for review! |
Bundle Size ReportSummary
Category Glance Per-category breakdownApp Entry Points — 3.33 MB (baseline 3.33 MB) • 🔴 +738 BMain entry bundles and manifests
Status: 2 added / 2 removed Graph Workspace — 724 kB (baseline 724 kB) • 🔴 +88 BGraph editor runtime, canvas, workflow orchestration
Status: 1 added / 1 removed Views & Navigation — 8.14 kB (baseline 8.14 kB) • ⚪ 0 BTop-level views, pages, and routed surfaces
Status: 1 added / 1 removed Panels & Settings — 294 kB (baseline 294 kB) • ⚪ 0 BConfiguration panels, inspectors, and settings screens
Status: 6 added / 6 removed UI Components — 12.3 kB (baseline 12.3 kB) • ⚪ 0 BReusable component library chunks
Status: 1 added / 1 removed Data & Services — 10 kB (baseline 10 kB) • ⚪ 0 BStores, services, APIs, and repositories
Status: 1 added / 1 removed Utilities & Hooks — 1.07 kB (baseline 1.07 kB) • ⚪ 0 BHelpers, composables, and utility bundles
Vendor & Third-Party — 5.36 MB (baseline 5.36 MB) • ⚪ 0 BExternal libraries and shared vendor chunks
Other — 2.55 MB (baseline 2.55 MB) • ⚪ 0 BBundles that do not match a named category
|
arjansingh
approved these changes
Oct 31, 2025
|
@benceruleanlu Backport to Please manually cherry-pick commit Conflicting files
|
benceruleanlu
added a commit
that referenced
this pull request
Nov 1, 2025
…e metrics (#6476) - Adds app:workflow_opened and plumbs open_source across drag/drop, file-open button, workspace, and templates - Tracks missing_node_count and missing_node_types for both workflow_opened and workflow_imported - Reuses WorkflowOpenSource type for consistency; no breaking changes to loadGraphData callers (5th param remains options object; openSource optional) Validation - pnpm lint:fix - pnpm typecheck Notes - Telemetry only runs in cloud builds; OSS remains clean. - loadGraphData telemetry is centralized where missing_node_types is computed. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-6476-feat-telemetry-add-workflow_opened-with-open_source-and-missing-node-metrics-29d6d73d365081f385c0da29958309da) by [Unito](https://www.unito.io) --------- Co-authored-by: bymyself <[email protected]>
benceruleanlu
added a commit
that referenced
this pull request
Nov 1, 2025
…e metrics (#6476) - Adds app:workflow_opened and plumbs open_source across drag/drop, file-open button, workspace, and templates - Tracks missing_node_count and missing_node_types for both workflow_opened and workflow_imported - Reuses WorkflowOpenSource type for consistency; no breaking changes to loadGraphData callers (5th param remains options object; openSource optional) Validation - pnpm lint:fix - pnpm typecheck Notes - Telemetry only runs in cloud builds; OSS remains clean. - loadGraphData telemetry is centralized where missing_node_types is computed. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-6476-feat-telemetry-add-workflow_opened-with-open_source-and-missing-node-metrics-29d6d73d365081f385c0da29958309da) by [Unito](https://www.unito.io) --------- Co-authored-by: bymyself <[email protected]>
This was referenced Nov 1, 2025
christian-byrne
added a commit
that referenced
this pull request
Nov 1, 2025
… metrics (#6476) (#6497) Backport of #6476 onto rh-test. - Adds telemetry events for `workflow_opened` and `workflow_imported` including `open_source` and missing node metrics. - Resolves merge conflict in `src/scripts/app.ts` by keeping the telemetry block after `afterConfigureGraph`. - Includes template load and file input changes to pass `openSource`. Files changed: - src/platform/telemetry/providers/cloud/MixpanelTelemetryProvider.ts - src/platform/telemetry/types.ts - src/platform/workflow/templates/composables/useTemplateWorkflows.ts - src/scripts/app.ts - src/scripts/ui.ts Validated with `pnpm lint:fix` and `pnpm typecheck`. ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-6497-Backport-telemetry-workflow_opened-with-open_source-and-missing-node-metrics-6476-29e6d73d365081238b8cef1d1a44287f) by [Unito](https://www.unito.io) Co-authored-by: bymyself <[email protected]>
christian-byrne
pushed a commit
that referenced
this pull request
Nov 1, 2025
…6501) Backport of upstream PR #6499 onto `rh-test`. Summary - Adds unified telemetry event `app:run_triggered` with `{ trigger_source: 'button' | 'keybinding' | 'menu' }`. - Instruments all run initiation paths: - Queue button emits `run_triggered` (source `button`) and still emits `run_button_click` for UI-only tracking. - Keybindings emit `run_triggered` (source `keybinding`). - Menus (menubar + legacy menu buttons) emit `run_triggered` (source `menu`). - Mixpanel provider implements `trackRunTriggered`. - No changes to `execution_start` logic. Files changed (matching PR #6499 exactly) - src/components/actionbar/ComfyRunButton/ComfyQueueButton.vue - src/platform/telemetry/providers/cloud/MixpanelTelemetryProvider.ts - src/platform/telemetry/types.ts - src/scripts/ui.ts - src/services/keybindingService.ts - src/stores/menuItemStore.ts Notes - Strictly limited to PR #6499; does NOT include unrelated changes (e.g., PR #6476 `workflow_opened`). - Local pre-push hook (knip) flagged an exported type as unused; pushed with `--no-verify` to avoid adding non-PR changes. Lint and typecheck pass locally (`pnpm lint:fix && pnpm typecheck`). Upstream reference: #6499 ┆Issue is synchronized with this [Notion page](https://www.notion.so/PR-6501-backport-pr-6499-unified-app-run_triggered-event-onto-rh-test-29e6d73d36508122ab3df5296e544b03) by [Unito](https://www.unito.io) --------- Co-authored-by: GitHub Action <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Validation
Notes
┆Issue is synchronized with this Notion page by Unito