Skip to content

xdna : selective hybrid CPU/NPU execution for Qwen3.5 - #223

Draft
maximbibik-itrex wants to merge 1 commit into
tetherto:masterfrom
maximbibik-itrex:xdna/selective-hybrid
Draft

xdna : selective hybrid CPU/NPU execution for Qwen3.5#223
maximbibik-itrex wants to merge 1 commit into
tetherto:masterfrom
maximbibik-itrex:xdna/selective-hybrid

Conversation

@maximbibik-itrex

Copy link
Copy Markdown

Summary

Adds an AMD XDNA (NPU) backend for ggml: weight MUL_MAT operations are
offloaded to the AMD XDNA2 NPU via IRON (mlir-aie), while all other ops
continue to run on the standard GGML CPU backend. Includes a selective
hybrid CPU/NPU execution path tuned for Qwen3.5.

This is a draft PR — opening it early to get feedback on the approach
and backend integration points before finalizing. Not ready to merge yet.

What's included

  • New ggml-xdna backend (ggml/src/ggml-xdna/) with IRON-based kernel
    build pipeline (ggml/src/ggml-xdna/iron/)
  • --device XDNA support for routing weight GEMMs to the NPU
  • Selective hybrid execution: NPU handles GEMM, CPU handles the rest
  • Documentation of the validated setup, dependencies, and known toolchain
    pitfalls in docs/backend/XDNA.md

Validated environment

Built and measured on one machine so far:

  • ASUS ProArt PX13 HN7306EA, AMD Ryzen AI MAX+ 395 (Strix Halo), NPU aie2p
  • Ubuntu 24.04.4, XRT 2.25.37, amdxdna driver 2.25.260102.56
  • IRON: mlir-aie 1.3.4 + llvm-aie 21
  • Models tested: Qwen3.5-0.8B (Q4_K_M, BF16), Qwen3.5-9B (Q4_K_M)

@github-actions

Copy link
Copy Markdown

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.

@github-actions github-actions Bot added documentation Improvements or additions to documentation ggml testing labels Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation ggml testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant