[skyrl-train] Update Megatron + LoRA to use PEFT only bridge#885
Draft
erictang000 wants to merge 9 commits intoNovaSky-AI:mainfrom
Draft
[skyrl-train] Update Megatron + LoRA to use PEFT only bridge#885erictang000 wants to merge 9 commits intoNovaSky-AI:mainfrom
erictang000 wants to merge 9 commits intoNovaSky-AI:mainfrom
Conversation
…nto megatron_lora_bridge
erictang000
added a commit
that referenced
this pull request
Jan 17, 2026
…lm and mcore extras (#887) Upgrading vllm to latest (minor changes). Needed for #885 and should subsume #882 Upgrades transformer-engine from 2.9.0 -> 2.10.0 for the megatron backend due to incompatibility with triton 3.5 (required by torch 2.9.0) Keeps the vllm_engine.py path backwards compatible for versions of vllm < 0.11.2 (important for flash-rl integration)
dzorlu
pushed a commit
to fleet-ai/SkyRL
that referenced
this pull request
Feb 4, 2026
…lm and mcore extras (NovaSky-AI#887) Upgrading vllm to latest (minor changes). Needed for NovaSky-AI#885 and should subsume NovaSky-AI#882 Upgrades transformer-engine from 2.9.0 -> 2.10.0 for the megatron backend due to incompatibility with triton 3.5 (required by torch 2.9.0) Keeps the vllm_engine.py path backwards compatible for versions of vllm < 0.11.2 (important for flash-rl integration)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After this Megatron-Bridge PR: NVIDIA-NeMo/Megatron-Bridge#1766
We should update LoRA + Megatron to only export LoRA weights, and to enable LoRA on vllm with megatron. This should help with making weight syncing for LoRA more efficient, and help reduce trainer/inference mismatch for LoRA.
TODOs:
LRUCacheWorkerLoRAManager- verl) to allow for in memory LoRA weight loading