v0.5.13

What's Changed

• chore: fetch transformers version from lightning-thunder requirements, for gpu test by @bhimrazy in #2184
• build(deps): update huggingface-hub requirement from >=0.30,<0.35 to >=0.30,<1.3 by @dependabot[bot] in #2139
• build(deps-dev): bump litdata from 0.2.51 to 0.2.59 by @dependabot[bot] in #2176
• [pre-commit.ci] pre-commit suggestions by @pre-commit-ci[bot] in #2180
• Doc: add uv options advanced installation fix by @sanggusti in #2185
• Fix rope_cache_length computation for models with rope_local_base_freq by @adi776borate in #2179
• feat: add generate_strategy option to litgpt serve by @adi776borate in #2188
• fix: gemma-3 checkpoint conversion from litgpt to hf by @adi776borate in #2195
• Allow train.max_steps for profiling runs by @MukeshK17 in #2193
• build(deps-dev): update bitsandbytes requirement from <0.43,>=0.42 to >=0.42,<0.50 by @dependabot[bot] in #2199
• build(deps): update huggingface-hub requirement from <1.3,>=0.30 to >=0.30,<1.4 by @dependabot[bot] in #2200
• feat(logger) : Add LitLogger Support for Lightning.ai Experiment Tracking by @bhimrazy in #2198
• fix: correct LoRA index computation uses wrong feature size by @Mr-Neutr0n in #2201
• fix: default val_split_fraction to 0.05 for single-file JSON data by @dario-fumarola in #2204
• fix: flaky test_litgpt_chat_endtoend by @bhimrazy in #2209
• chore(workflows): Sync GPU CI workflow image with lightning-thunder repo by @bhimrazy in #2208
• Add YaRN rope adjustment + DeepSeekV3 rope_interleave by @ysjprojects in #2202
• fix: handle empty prompt list in R1Base.apply() by @LincolnBurrows2017 in #2216
• Added correct rope_base to all Mistral configs by @adi776borate in #2218
• chore(ruff): bump target-version to py310 and update type annotations by @bhimrazy in #2221
• fix: raise error when prefill exceeds sliding window KV cache size by @alvinttang in #2215
• LoRAQKVLinear.lora_ind does not cache index by @mseeger in #2225
• feat: Introduce a pre-flight validation script for LitGPT checkpoints… by @OnePunchMonk in #2214
• build(deps): bump psutil from 7.1.3 to 7.2.2 by @dependabot[bot] in #2226
• [pre-commit.ci] pre-commit suggestions by @pre-commit-ci[bot] in #2229
• fix(finetune): run generate_example only on rank 0 in multi-GPU training by @Br1an67 in #2213
• fix(tests): Fix 3 test failures introduced by the validate PR by @bhimrazy in #2233
• fix: make sample() produce greedy decoding when temperature=0 by @CrepuscularIRIS in #2239
• fix(tests): xfail DynamoThunder tests with torch>=2.12 by @bhimrazy in #2254
• security: remove pull_request_target trigger from CPU tests workflow by @bhimrazy in #2253
• chore: pin GitHub Actions to verified commit SHAs by @bhimrazy in #2256
• ci: migrate workflows from pip to uv by @bhimrazy in #2257
• fix(ci): unblock check-links and stabilize GPU serve tests by @bhimrazy in #2270
• fix(tests): skip tokenizer tests for only gated model in fork prs by @bhimrazy in #2268
• Omit ChatML system turn when there is no system message (Qwen3 emitted literal 'None') by @lollinng in #2258
• fix(tests): use Gutenberg mirror to fix flaky book downloads by @bhimrazy in #2272
• docs: add Code of Conduct and rename LICENSE filename by @bhimrazy in #2273
• ci: migrate PyPI release to trusted publishing (OIDC) by @bhimrazy in #2261
• docs: add CONTRIBUTING.md by @bhimrazy in #2276
• docs: update CODEOWNERS with patterns and retired committers by @bhimrazy in #2275
• fix(docs): update broken links in README by @bhimrazy in #2277
• chore: bump version to 0.5.13 by @bhimrazy in #2279

New Contributors

• @sanggusti made their first contribution in #2185
• @adi776borate made their first contribution in #2179
• @MukeshK17 made their first contribution in #2193
• @Mr-Neutr0n made their first contribution in #2201
• @dario-fumarola made their first contribution in #2204
• @LincolnBurrows2017 made their first contribution in #2216
• @alvinttang made their first contribution in #2215
• @OnePunchMonk made their first contribution in #2214
• @Br1an67 made their first contribution in #2213
• @CrepuscularIRIS made their first contribution in #2239
• @lollinng made their first contribution in #2258

Full Changelog: v0.5.12...v0.5.13

v0.5.12

What's Changed

• fix: update TinyStories optimize to use item_loader=TokensLoader() by @bhimrazy in #2148
• [FEAT] Add Grouped Topk Routing to LLaMAMoE (Based on DeepseekV3MoE) by @ysjprojects in #2134
• Add timeout parameter to run_server. by @hv10 in #2157
• fix: chat prompt lowercasing bug by @bhimrazy in #2155
• chore: Drop Python 3.9 support from ci by @bhimrazy in #2161
• fix: handle gemma 3 weights prefix during hf conversion by @scascar in #2156
• Fix to avoid RuntimeError by @mseeger in #2138
• fix: pin lm-eval<0.4.9.1 for trust_remote_code issue by @bhimrazy in #2168
• Small refactorings by @mseeger in #2166
• docs: enhance Python API tutorial with fixes and new section by @carsch18 in #2169
• fix: sliding window KV cache for Gemma-3 models (issue #2145) by @Chelsi-create in #2153

New Contributors

• @lianakoleva made their first contribution in #2132
• @hv10 made their first contribution in #2157
• @scascar made their first contribution in #2156
• @carsch18 made their first contribution in #2169
• @Chelsi-create made their first contribution in #2153

Full Changelog: v0.5.11...v0.5.12

v0.5.11

What's Changed

• Add Qwen3 2507 Thinking models by @ysjprojects in #2110
• Add MLA by @ysjprojects in #2113
• Add Qwen3 2507 Instruct models by @ysjprojects in #2125
• finetune_lora upgrades by @ysjprojects in #2086
• Use torch autograd integration for backward testing by @t-vi in #2123
• bump PT 2.8 & install with --upgrade-strategy greedy by @Borda in #2107
• build(deps): update transformers requirement from <4.54,>=4.51.3 to >=4.51.3,<4.57 by @dependabot[bot] in #2120
• build(deps): update huggingface-hub requirement from <0.33,>=0.30 to >=0.30,<0.35 by @dependabot[bot] in #2119
• build(deps): update bitsandbytes requirement from <0.43,>=0.42 to >=0.42,<0.47 by @dependabot[bot] in #2069

---

Full Changelog: v0.5.10...v0.5.11

v0.5.10

What's Changed

• qwen2.5 long context by @ysjprojects in #1933
• debug installing torch for Thunder by @Borda in #2072
• Fix in convert_hf_checkpoint related to Gemma 3 by @mseeger in #2062
• debug some failing standalone tests with compiler by @Borda in #2075
• Qwen3 MoE by @ysjprojects in #2060
• Deferring import of torch in config to allow faster import by @JackUrb in #2079
• update bug-report/issue with reproducing in Studio by @Borda in #2081
• ci: show the longest tests for improvement by @Borda in #2083
• docs: Add documentation for OpenAI-compatible API in LitGPT deployment by @bhimrazy in #2082
• doc: add n_query_groups to attention notation table by @raishish in #2092
• doc: add comments for clarifying query / KV groups by @raishish in #2093
• Complete pending todos in testing by @raishish in #2088
• pin: restrict datasets version to <4.0.0 for compatibility by @Borda in #2095
• add/debug Lit CI by @Borda in #2094
• add api_path parameter to cli options to allow custom API endpoint configuration by @botirk38 in #2080
• Fix commands in pretrain.md by @H4kor in #2097

New Contributors

• @JackUrb made their first contribution in #2079
• @raishish made their first contribution in #2092
• @botirk38 made their first contribution in #2080
• @H4kor made their first contribution in #2097

Full Changelog: v0.5.9...v0.5.10

v0.5.9

What's Changed

• input_pos_maxp1 as a Python integer by @Andrei-Aksionov in #2016
• add testing for py3.12 & py3.13 by @Borda in #2025
• ci: update guardian for PRs by @Borda in #2043
• Add devcontainer by @twsl in #2035
• Remove dependency from config to utils by @lukemerrick in #2034
• fix: Add fallback chat template by @andyland in #2040
• Add optional sys prompt by @twsl in #2036
• fix: Pretraining text files with recent litdata versions by @andyland in #2048
• tests: mark test_evaluate_script as flaky by @Borda in #2049
• simplify the GPU testing flow by @Borda in #2053
• ci: extend testing with ubuntu-24.04 by @Borda in #2056
• Remove litserve version constraint by @twsl in #2055
• req: pin bitsandbytes>=0.45.2,<0.45.5 by @Borda in #2057
• ci: use Thunder dev images for testing by @Borda in #2054
• Update spacing in README.md by @Borda in #2058
• Transformers version bump by @KaelanDt in #2029
• Qwen3 Dense by @ysjprojects in #2044
• phi-4 reasoning models by @ysjprojects in #2047
• adding logger args by @ysjprojects in #1973
• Qwen3 MoE Preliminary: add intermediate_size argument to MLP modules by @ysjprojects in #2046
• OLMo 2 by @ysjprojects in #1897
• bump: testing with PT 2.7.1 by @Borda in #2063
• Add Dependabot for Pip & GitHub Actions by @Borda in #2066
• build(deps): bump litdata from 0.2.45 to 0.2.49 by @dependabot in #2068

New Contributors

• @twsl made their first contribution in #2035
• @lukemerrick made their first contribution in #2034
• @prabhuteja12 made their first contribution in #2027
• @andyland made their first contribution in #2040
• @KaelanDt made their first contribution in #2029

Full Changelog: v0.5.8...v0.5.9

v0.5.8

Many great updates!

What's Changed

• add missing r1 prompt style by @ali-alshaar7 in #1929
• fix incremental save for PyTorch 2.6 by @t-vi in #1928
• fix: resolve failing CI by @Borda in #1944
• handle wrapped thundermodules in generate by @t-vi in #1955
• fix skip condition by @t-vi in #1956
• ci: use HF cache by @Borda in #1958
• ci: split HF caching by @Borda in #1960
• bump: PT 2.6 + bitsandbytes & standalone tests by @Borda in #1959
• prune whitespaces for code readability by @Borda in #1962
• fixing various typos in examples & tutorials by @Borda in #1963
• fix n_query_groups for llama-3.1-405b by @ysjprojects in #1946
• tests: make flaky test due to connection issues by @Borda in #1964
• Fix: incorrect gradient accumulation steps bug by @ysjprojects in #1947
• fix: use default num_nodes=1 for back-compatibility by @Borda in #1967
• Do not wrap LoRA layers with FSDP by @janEbert in #1538
• Speculative decoding: Base implementation by @Andrei-Aksionov in #1938
• Better clarity on SFT dataset attributes by @ysjprojects in #1970
• Enforce Consistent Formatting and Validation for YAML Files by @Borda in #1977
• Apply Standard Formatting and Fix Import & Test Name Issues by @Borda in #1981
• Setting config.sliding_window_layer_stride explicity by @ysjprojects in #1972
• feat: add linear rope type by @k223kim in #1982
• feat: update tests for transformers 4.50.2 by @k223kim in #1983
• fix: test_tokenizer_against_hf by @Borda in #1984
• feat: replace sliding window type with offset by @k223kim in #1989
• ci: with pull_request_target by @Borda in #1992
• Phi4 mini by @ysjprojects in #1949
• aggregate val_loss by @ysjprojects in #1971
• feat: add local base freq for rope by @k223kim in #1993
• test: flexible wait for serve start by @Borda in #1996
• fix: replace sliding window configuration parameters to sliding windows indices by @k223kim in #1995
• QwQ-32B by @ysjprojects in #1952
• feat: run thunder tests as part of LitGPT CI by @deependujha in #1975
• try pyupgrade-up py38 by @Borda in #1999
• [1/4] feat: add gemma 3 27b by @k223kim in #1998
• [2/4] add gemma 3 1b by @k223kim in #2000
• [3/4] feat: add gemma 3 4b by @k223kim in #2001
• feat: add gemma-3-12b by @k223kim in #2002
• Add resume for adapter_v2, enable continued finetuning for adapter by @altria-zewei-wang in #1354
• Fix/loading gemma 3 1b by @pquadri in #2004
• feat: add gemma 3 in readme and tutorials by @k223kim in #2005
• add borda as codeowner by @t-vi in #2007
• example for full finetuning with python code by @astrobdr in #1331
• feat: add tests for gemma3 by @k223kim in #2006
• building tutorials as mkdocs pages by @Borda in #2011
• Add mlflow logger support by @topikachu in #1985
• fix support for litserve>0.2.4 by @ali-alshaar7 in #1994
• Cast tensors in KVCache only when needed by @Andrei-Aksionov in #2017
• feat: load only text weights from multimodal gemma by @pquadri in #2008
• Feature: Adds support for OpenAISpec in litgpt serve by @bhimrazy in #1943
• drop upper bounds in dependencies by @t-vi in #2022

New Contributors

• @syntheticgio made their first contribution in #1939
• @deependujha made their first contribution in #1975
• @altria-zewei-wang made their first contribution in #1354
• @pquadri made their first contribution in #2004
• @astrobdr made their first contribution in #1331
• @topikachu made their first contribution in #1985
• @bhimrazy made their first contribution in #1943
• @Lynsoo made their first contribution in #2018

Full Changelog: v0.5.7...v0.5.8

v0.5.7

What's Changed

• Add Deepseek r1 distill llama models by @ali-alshaar7 in #1922

New Contributors

• @ali-alshaar7 made their first contribution in #1922

Full Changelog: v0.5.6...v0.5.7

v0.5.6

What has changed

• Phi-4 by @ysjprojects
• Thunder compatibility by @t-vi

v0.5.5

What's Changed

• Post-release setup for 0.5.5.dev1 by @Andrei-Aksionov in #1885
• Falcon3 by @ysjprojects in #1881
• ChatML prompt template by @ysjprojects in #1882
• Small fixes and refactoring by @mseeger in #1861
• Drop interleave placement in QKV matrix by @Andrei-Aksionov in #1013
• Bump PyTorch, PyTorch-Lightning and BnB versions by @Andrei-Aksionov in #1893
• Pin version of mistune in check links workflow by @Andrei-Aksionov in #1895
• Skip converting .safetensors to .bin by @ysjprojects in #1853
• Some improvements for KV caching by @mseeger in #1891
• added query-key norm to accomodate OLMo2 by @ysjprojects in #1894
• Improve HF download speed by @rasbt in #1899
• Bump version for 0.5.5 release by @rasbt in #1901

New Contributors

• @mseeger made their first contribution in #1861

Full Changelog: v0.5.4...v0.5.5

v0.5.4

What's Changed

• 0.5.3 post release setup by @rasbt in #1817
• Add cff file by @rasbt in #1818
• Deprecate Support for Dolly, Nous-Hermes, Redpajama-Incite, Vicuna and H2O Danube Models. by @ParagEkbote in #1821
• Adding OLMo by @aflah02 in #1827
• Adding Qwen2.5 by @ysjprojects in #1834
• Restore SlimPajama preprocessing code by @aflah02 in #1840
• Add QwQ-32B-Preview by @ysjprojects in #1844
• Add Mixtral-8x22B by @ysjprojects in #1845
• add Llama-3.3-70B-Instruct by @ysjprojects in #1859
• add Salamandra by @ysjprojects in #1857
• Qwen2.5: fix block size for Coder series by @ysjprojects in #1856
• fix: add missing"," by @vra in #1855
• fix llama3.3 readme url by @ysjprojects in #1862
• Set torch.load(..., weights_only=False) in litgpt/api.py by @Andrei-Aksionov in #1874
• Add Qwen2.5 math by @ysjprojects in #1863
• Add SmolLM2 by @ysjprojects in #1848
• Add Mistral-Large-Instruct-2411 by @ysjprojects in #1876
• Bump version for 0.5.4 release by @Andrei-Aksionov in #1883
• Temporary remove Thunder to make a release by @Andrei-Aksionov in #1884

New Contributors

• @ParagEkbote made their first contribution in #1821
• @ysjprojects made their first contribution in #1834
• @vra made their first contribution in #1855

Full Changelog: v0.5.3...v0.5.4