move unpack out of the flux, krea2 and ltx decoders into the core denoise groups - #14737
Open
akshan-main wants to merge 5 commits into
Open
move unpack out of the flux, krea2 and ltx decoders into the core denoise groups#14737akshan-main wants to merge 5 commits into
akshan-main wants to merge 5 commits into
Conversation
…eaves in the state
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes #14730.
Adds a test to the modular tester mixin that checks the shape of the latents each pipeline leaves in the state after denoising, and every tester now declares
expected_latents_shape. Running it across all families catches the three from the issue, plus ltx2 (being moved in #14612, so it's declared at its current packed shape until then) and wan_animate_2, which decodes inside its segment loop and never leaves latents behind, so it opts out withlatents_output_name = None.For flux, krea2 and ltx the unpack now happens at the end of the core denoise group, same shape as
Flux2UnpackLatentsStep. The decoders just denormalize and decode. They still accept packed latents with a deprecation warning, which is why height/width (and num_frames for ltx) are still on them, and there's a test per family for that path.Outputs are identical to main on the test repos and on the real checkpoints (images in akshan-main/latents-form-verification). Loading LTX-Video with
ModularPipeline.from_pretrainedneeds #14736. Also wrote the principle down in.ai/references/modular.mdin a separate commit; feel free to drop it or change it.Before submitting
Who can review?
@yiyixuxu