Skip to content

Add xDiT kernel source mapping support - #1154

Open
kyle-hoffmeyer wants to merge 14 commits into
mainfrom
khoffmey/feat/op_source_mapping_enhancements
Open

Add xDiT kernel source mapping support#1154
kyle-hoffmeyer wants to merge 14 commits into
mainfrom
khoffmey/feat/op_source_mapping_enhancements

Conversation

@kyle-hoffmeyer

@kyle-hoffmeyer kyle-hoffmeyer commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR adds support for kernel source mapping for xDiT diffusion workloads. Each fix addresses a specific failure observed during source resolution of device kernels in these traces.

Change 1: Replace c++filt with itanium-demangler

File: source_resolver.py_demangle_itanium() replacing _cxxfilt_base()

Problem: c++filt is the tool used to demangle kernel names. An example of a mangled kernel name:

_ZN2ck16tensor_operation6device12_GLOBAL__N_139kernel_grouped_conv_fwd_xdl_cshuffle_v3I...

These kernel names are very difficult to parse but can be demangled to more easily parse.
c++filt (GNU Binutils) fails on xDiT workloads due to deeply nested CK convolution kernel symbols (~20+ levels of template nesting, 1575+ characters). When demangling fails, c++filt returns the input string unchanged — an ambiguous failure signal that causes _non_patchable_kind() to misclassify CK kernels as patchable.

itanium-demangler handles deeper nesting (no hardcoded recursion limits) and returns "" on failure (unambiguous). This makes the existing if demangled: guard in _non_patchable_kind() work correctly — "" is falsy, so the mangled regex fallback fires and CK kernels are correctly classified as aiter_ck.

xDiT example: aten::miopen_convolution dispatches to CK grouped convolution:

_ZN2ck16tensor_operation6device12_GLOBAL__N_139kernel_grouped_conv_fwd_xdl_cshuffle_v3I... (1575 chars)

c++filt:            returns input unchanged (failure, no signal)
                    → if demangled: True → searches "ck::" in mangled string → no match
                    → mangled regex fallback never reached
                    ✗ CK kernel misclassified as patchable

itanium-demangler:  returns "" (clear failure signal)
                    → if demangled: False → falls through
                    → mangled regex fires → matches "ck"
                    ✓ CK kernel classified as non-patchable

14 CK convolution kernels across 4 xDiT traces are affected.

Comparison:

c++filt (GNU Binutils) itanium-demangler (Python)
Deeply nested CK templates (~20 levels) Fails (returns input) Handles them
Failure signal Returns input unchanged (ambiguous) Returns "" (clear)
Dependencies Requires binutils installed in container pip package, no system deps
Performance Subprocess spawn per call (cached) Pure Python, in-process (cached)
Imports required shutil, subprocess None (graceful ImportError fallback)

Change 2: Fix _base_from_demangled() for (anonymous namespace):: kernels

File: source_resolver.py_base_from_demangled()

Problem: _base_from_demangled() is the function used to parse a demangled kernel name. _base_from_demangled() used re.split(r"[(<]", ...) to extract the kernel name. For signatures containing (anonymous namespace), the ( in the namespace qualifier was matched first, truncating the name to "void".

Fix: Rewrite _base_from_demangled() to match the logic in _bypass_source_resolver._demangle_kernel_name(), which already handles this case correctly. Both functions now use the same approach for extracting the bare function identifier: strip void prefix, strip (anonymous namespace)::, then remove template/function args. Ideally this shared logic would live in a common utility function, but that belongs to a broader cleanup PR.

Change 3: Add Tensile and MIOpen non-patchable classification

File: source_resolver.py_non_patchable_kind()

Problem: Tensile GEMM kernels (Cijk_*) and MIOpen convolution kernels were reported as "unresolved" with no explanation. These kernels are written in GPU assembly and shipped pre-compiled — no source code exists.

Fix: _non_patchable_kind() now accepts an op_name parameter. Tensile is identified by device kernel name prefix (Cijk_*). MIOpen is identified by op name ("miopen" in op_name) rather than device kernel name patterns, which vary across MIOpen versions (e.g., igemm_*, batched_transpose_*, Im2d2Col, Im3d2Col, SubTensorOpWithScalar*). Using the op name catches all MIOpen kernels regardless of the device kernel naming convention.

xDiT examples:

Tensile GEMMs (~37% of xDiT device kernels):

aten::mm       → Cijk_Ailk_Bljk_SBB_BFP16_BFP16_BFP16_...   → tensile_precompiled
aten::addmm    → Cijk_Ailk_Bljk_SB_BFP16_BFP16_BFP16_...    → tensile_precompiled
aten::bmm      → Cijk_Alik_Bljk_SB_BFP16_BFP16_BFP16_...    → tensile_precompiled

MIOpen convolutions (all caught via op name aten::miopen_convolution):

aten::miopen_convolution → igemm_fwd_gtcx3_nchw_fp16_...     → miopen_precompiled
aten::miopen_convolution → batched_transpose_b_n_c_h_w_...   → miopen_precompiled
aten::miopen_convolution → Im3d2Col                           → miopen_precompiled

These are now reported as "non_patchable" with a reason string instead of bare "unresolved".

Change 4: Mark Inductor-generated Triton kernels as non-patchable

File: _bypass_source_resolver.pyresolve_triton_py()
File: _bypass_report.py

Problem: torch.compile Inductor-generated Triton kernels are not patchable as the source for them is generated at compile time within Inductor. These kernels were previously reported as "unresolved" from resolve_triton_py. Tier 1 correctly identified them as non-editable (via torchinductor in the kernel_file path), but returned "unresolved" instead of "non_patchable", causing tiers 2 and 3 to run unnecessarily.

Fix: When resolve_triton_py() has a kernel_file path but editable_trace_source() rejects it, return "non_patchable" instead of "unresolved". The framework scan/repo scan gates in _bypass_report.py now use source_method == "unresolved", so only genuinely unresolved kernels fall through to tiers 2 and 3.

xDiT example: Inductor-generated Triton kernels:

triton_poi_fused_addmm_cat_gelu_slice_view_25
  kernel_file: /tmp/torchinductor_root/4o/c4odgozyt6ydnxpdsf...
  Before: "unresolved" (framework + repo scan ran, found nothing)
  After:  "non_patchable" (framework + repo scan skipped)

Verification

Tested via the full pipeline (analyze_trace(top_k=0)build_candidates(top_k=0)) in tracelens-xdit:v26.7 (TraceLens patched xDiT image) against 4 xDiT diffusion workloads. Traces can be found in xDiT folders in https://github.com/AMD-AGI/TraceLens/tree/main/tests/traces/inference.

Kernel type Before After
Tensile GEMM (Cijk_*) unresolved non_patchable (tensile_precompiled)
MIOpen IGEMM (igemm_*) unresolved non_patchable (miopen_precompiled)
MIOpen helpers (batched_transpose_*) unresolved non_patchable (miopen_precompiled)
CK conv (_ZN2ck...) unresolved (c++filt failed silently) non_patchable (aiter_ck)
Inductor Triton (triton_poi_*) unresolved (framework + repo scan wasted) non_patchable (framework + repo scan skipped)
AITER groupnorm unresolved (_base_from_demangled returned "void") resolvedgroupnorm.cu
AITER/Triton attention resolved resolved (unchanged)

New dependency

itanium-demangler (pip package, pure Python, no system deps). Added to pyproject.toml runtime optional dependencies. When not installed, a warning is logged ("itanium-demangler is not installed. Kernel classification may be degraded.") and demangling falls back to the mangled regex path.

@kyle-hoffmeyer
kyle-hoffmeyer force-pushed the khoffmey/feat/op_source_mapping_enhancements branch from 37b4013 to c9b4fff Compare August 11, 2026 20:21
Comment thread src/hyperloom/agents/kernel/tools/source_resolver.py Fixed
Comment thread src/hyperloom/agents/kernel/tools/source_resolver.py Fixed
Base automatically changed from ahasssan/feat/Op_source_finding to main August 13, 2026 03:59
kyle-hoffmeyer and others added 3 commits August 14, 2026 13:40
…n kernels

- Replace c++filt subprocess with itanium-demangler Python package for
  reliable demangling of deeply-nested CK template symbols
- Fix base_symbol() returning "void" for (anonymous namespace):: kernels
  by delegating to _demangle_kernel_name()
- Add non-patchable classification for Tensile (Cijk_*) and MIOpen
  (igemm_*, batched_transpose_*, Im2d2Col*, SubTensorOpWithScalar*)
- Tighten CK guard to `if demangled and demangled != raw:` so mangled
  regex fallback fires when demangling returns input unchanged
- Expand scan paths: add .hpp to native extensions, remove /3rdparty/
  skip from repo scan, add 3rdparty/composable_kernel/include to
  _CSRC_DIRS for CK kernel discovery

Tested against 9 workloads (xDiT, vLLM, SGLang) across 3 containers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kyle-hoffmeyer
kyle-hoffmeyer force-pushed the khoffmey/feat/op_source_mapping_enhancements branch from 710d39a to d860f3a Compare August 14, 2026 17:51
kyle-hoffmeyer and others added 3 commits August 14, 2026 13:58
Remove vllm from _repo_scan_roots() and revert tier 3 ranking logic
back to original ambiguity-refusal behavior. These changes are for
vLLM/SGLang workloads and not needed for the xDiT focus of this branch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The `if demangled and demangled != raw` guard is not needed when using
itanium-demangler, which returns "" on failure (falsy). The original
`if demangled:` guard works correctly with the new demangler.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove 3rdparty/composable_kernel/include from _CSRC_DIRS, restore
/3rdparty/ to _SCAN_SKIP_MARKERS, and remove .hpp from native extension
lists. These changes only affect CK kernels which are classified as
aiter_ck (non_patchable) before any index lookup — the indexed source
is never returned.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 14, 2026

Copy link
Copy Markdown

CI E2E report — ✅ Succeeded

item value
result ✅ Succeeded
model Qwen/Qwen3-0.6B (dense)
resources 1× GPU, TP=1
PR branch khoffmey/feat/op_source_mapping_enhancements
commit d2857d49aba8a8bcc1b0c1aa133ce68d1b84207e
session_id 2b3b4188-bdf0-46a5-ae0c-393488a88481
queue → dispatch 0s
run time 178m 50s
total 178m 50s

details

@kyle-hoffmeyer kyle-hoffmeyer changed the title Khoffmey/feat/op source mapping enhancements Add xDiT kernel source mapping support Aug 17, 2026
@kyle-hoffmeyer
kyle-hoffmeyer marked this pull request as ready for review August 17, 2026 21:34
@kyle-hoffmeyer
kyle-hoffmeyer marked this pull request as draft August 17, 2026 21:35
@kyle-hoffmeyer
kyle-hoffmeyer marked this pull request as ready for review August 17, 2026 22:11
@kyle-hoffmeyer
kyle-hoffmeyer marked this pull request as draft August 17, 2026 22:16
@kyle-hoffmeyer

Copy link
Copy Markdown
Collaborator Author

I'm not sure if I added the itanium-demangler dependency correctly. Please take a look.

@kyle-hoffmeyer
kyle-hoffmeyer marked this pull request as ready for review August 18, 2026 23:11
@Ahmedhasssan-aig

Copy link
Copy Markdown
Collaborator

Can you please try to pass all the CI tests?

def _cxxfilt_base(mangled: str) -> str:
"""Demangle via ``c++filt`` when available (``""`` on failure).
def _demangle_itanium(mangled: str) -> str:
"""Demangle an Itanium-mangled symbol via ``itanium-demangler``.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we keep the _cxxfilt_base (to use it as fallback) if itanium is not installed or missing for any reason?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants