[not to land] Tiny Llama4 debug model, activation offloading repro

[ebsmothers]

No description provided.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchtune/2708

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 194cdf3 with merge base 3310946 ():

NEW FAILURE - The following job has failed:

• Lint / lint (3.10) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 11.53846% with 23 lines in your changes missing coverage. Please review.

Project coverage is 60.60%. Comparing base (113807c) to head (194cdf3).
Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
torchtune/models/llama4/_model_builders.py	13.33%	13 Missing ⚠️
torchtune/training/_activation_offloading.py	0.00%	5 Missing ⚠️
torchtune/training/_grad_scaler.py	20.00%	4 Missing ⚠️
recipes/full_finetune_distributed.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2708      +/-   ##
==========================================
- Coverage   63.37%   60.60%   -2.78%     
==========================================
  Files         425      425              
  Lines       25647    25785     +138     
==========================================
- Hits        16255    15627     -628     
- Misses       9392    10158     +766     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.