[Roadmap]: Mori PoC Roadmap

[TianDi101]

MORI-EP

• Intra-node dispatch / combine kernel
• High-bandwidth inter-node dispatch / combine kernel
• Low-latency inter-node dispatch / combine kernel
• Pybind & aiter integration
• Sglang integration
• FP8 support
• FP4 support

MORI-IO

• Read / write APIs
• Batch transfer APIs
• Session transfer APIs
• Multi-QP support
• Fault tolerant (Retry / TCP fallback)
• TCP / XGMI transport support
• IBGDA support
• GDS support

Vendor support

• Mellanox
• BRCM Thor2
• Pensando

Performance

• IBGDA core primitive benchmark & optimization
• P2P core primitive benchmark & optimization
• PoC for fused communication & computation kernels

Framework

• RDMA atomic ops
• Topology detection
• Barrier abstraction
• Signal abstraction
• Testing framework
• IBRC transport
• Refactor & better abstraction of core primitives

MORI-CCL
TBD

[learner321]

Hi, is there a link to the Sglang branch that supports Mori?

[maifeeulasad]

Are you guys looking for collaborators? Cause I want to contribute to this.

[learner321]

您的邮件已收到！！！