Juzheng Zhang, Jiacheng You, Ashwinee Panda, Tom Goldstein
📄 Paper | 💻 Code | 🤗 HuggingFace | COLM 2025
LoRI (LoRA with Reduced Interference) is a simple yet effective variant of LoRA for fine-tuning LLMs. It freezes the projection matrices A as random projections and sparsifies B using task-specific masks. LoRI significantly reduces the number of trainable parameters, preserves single-task performance, and minimizes cross-task interference during adapter merging and continual learning.
Create and activate a Conda environment:
conda create -n lori python=3.10 -y
conda activate lori
Clone the repository and install dependencies:
git clone https://github.com/juzhengz/LoRI.git
cd LoRI
pip install -r requirements.txt
LoRI is implemented using Fully Sharded Data Parallel and can be executed in a multi-GPU environment. We provide training scripts covering Natural language understanding (NLU), Code generation, Mathematical reasoning, and Safety alignment. These scripts support LLaMA-3-8B and Mistral-7B base models with adapter ranks of 32 and 64. Each script performs LoRI-D training, extracts sparse masks, continues with LoRI-S training at 90% sparsity, and evaluates on downstream tasks.
Training code generation adapters LoRI-D and LoRI-S on the CodeAlpaca dataset using LLaMA-3-8B with rank 32:
bash scripts/codealpaca_llama3_r_32.sh
scripts/codealpaca_*.sh— Code generation tasksscripts/gsm8k_*.sh— Mathematical reasoning tasksscripts/nlu_*.sh— Natural language understanding tasksscripts/saferpaca_*.sh— Safety alignment tasks
Pretrained LoRI adapters are available via our HuggingFace collection and can be loaded as follows:
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B")
adapter = PeftModel.from_pretrained(base_model, "tomg-group-umd/LoRI-S_code_llama3_rank_32")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Meta-Llama-3-8B")
LoRI-D and LoRI-S adapters are provided for code, math, NLU, and safety tasks, using LLaMA-3-8B and Mistral-7B models at ranks 32 and 64.
Use the following scripts for merging adapters:
Update the adapter paths in the scripts to point to either your own trained adapters or those available on HuggingFace.
Use these scripts to perform continual learning with LoRI:
Before running the scripts, set model_archive to the path of your trained LoRI-D safety adapter, or use the safety adapter from our HuggingFace collection.
model_archive=/path/to/your/lori-d/safety/adapter
Before running the scripts, set model_archive to the path of your LoRI-S safety adapter and set mask_path to the path of the sparse mask for the downstream task. Alternatively, you can use the safety adapter and the corresponding sparse mask from our HuggingFace collection.
model_archive=/path/to/your/lori-s/safety/adapter
mask_path=/path/to/your/lori-d/code/adapter/masks/0.9_mask.pt
LoRI supports a variety of base models and loss functions, which can be found in the config/model and config/loss directories of the repository. To add a new model or loss function, you can simply create a new .yaml file in the respective directory.
This project builds on the codebase of dpo-rlaif and incorporates code from lottery-ticket-adaptation. We evaluate code generation performance on HumanEval using the bigcode-evaluation-harness.
If you use LoRI in your work, please cite:
@article{zhang2025lori,
title={LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptation},
author={Zhang, Juzheng and You, Jiacheng and Panda, Ashwinee and Goldstein, Tom},
journal={arXiv preprint arXiv:2504.07448},
year={2025}
}
Feel free to reach out if you have any questions!