Skip to content

[bugfix mxpf4] Infer mxfp4 quantmethod from layer #824

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ZhiweiYan-96 wants to merge 4 commits into
base: dev/perf_sync_upstream_20251124
Choose a base branch
from
Open

[bugfix mxpf4] Infer mxfp4 quantmethod from layer #824

ZhiweiYan-96 wants to merge 4 commits into from

Conversation

@ZhiweiYan-96
Copy link

@ZhiweiYan-96 ZhiweiYan-96 commented Nov 27, 2025 •
edited by github-actions bot
Loading

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@ZhiweiYan-96 ZhiweiYan-96 changed the title [bugfx mxpf4] Infer mxfp4 quantmethod from layer [bugfix mxpf4] Infer mxfp4 quantmethod from layer Nov 27, 2025
zejunchen-zejun
zejunchen-zejun reviewed Nov 28, 2025
View reviewed changes
vllm/model_executor/layers/quantization/quark/quark_moe.py Outdated
a2_scale=None,
block_shape=None,
)
return mxfp4_w4a16_moe_quant_config(layer.w13_weight_scale,
Copy link

@zejunchen-zejun zejunchen-zejun Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leave some comment here for why not choosing ocp_mx_moe_quant_config

@ZhiweiYan-96
Add comments
7da5e5f 
Signed-off-by: ZhiweiYan-96 <[email protected]>
@tjtanaa
Copy link

tjtanaa commented Nov 28, 2025

@xuebwang-amd @haoyangli-amd @fxmarty-amd can you help to take a look at this quark bugfix?
@fxmarty-amd I saw that in your PR vllm-project@41f1cf3#diff-c73528091f2176f6547e24074950962deec4311315fa5e99ef2ca7682680708c introduced ocp_mx_moe_quant_config. Do you intend ocp_mx_moe_quant_config to only support act-mxfp4 and weight-mxfp4, or should we extend the ocp_mx_moe_quant_config to support act-bfloat16 and weight-mxfp4.

@zhuyuhua-v zhuyuhua-v mentioned this pull request Nov 28, 2025
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zejunchen-zejun zejunchen-zejun zejunchen-zejun left review comments

@wuhuikx wuhuikx Awaiting requested review from wuhuikx wuhuikx is a code owner

@tjtanaavllm tjtanaavllm Awaiting requested review from tjtanaavllm tjtanaavllm is a code owner

@kliuae-amd kliuae-amd Awaiting requested review from kliuae-amd kliuae-amd is a code owner

@lihaoyang-amd lihaoyang-amd Awaiting requested review from lihaoyang-amd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ZhiweiYan-96 @tjtanaa @zejunchen-zejun