Unpaired Modality-Agnostic Generative Recommendation

This repository contains the official source code for the paper "Unpaired Modality-Agnostic Generative Recommendation."

Setup

cd /path/to/UnpairGR
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Data Preparation

We follow the Amazon 2018 preprocessing convention used by MQL4GRec. Refer to its data_process directory for the original download and feature-extraction scripts.

1. Download the Amazon 2018 ratings, metadata, and product images.
2. Keep items with both usable text metadata and an image, apply 5-core filtering, remap user/item IDs, and sort each user's interactions chronologically. Use leave-one-out splitting: the last interaction is for testing and the second-to-last interaction is for validation.
3. Encode each item's title and description with huggyllama/llama-7b using mean pooling, and save the features as {dataset}.emb-llama-td.npy.
4. Encode one image per item with CLIP ViT-L/14, and save the features as {dataset}.emb-ViT-L-14.npy.

Prepare the three target datasets (Arts, Games, and Instruments) and the six pre-training datasets (Pet, Cell, Automotive, Tools, Toys, and Sports) with the following layout:

data/
└── {dataset}/
    ├── {dataset}.inter.json
    ├── {dataset}.item.json
    ├── {dataset}.emb-llama-td.npy
    └── {dataset}.emb-ViT-L-14.npy

Item IDs must be zero-based and contiguous. Row i in both embedding arrays must correspond to item ID i, and the two arrays must contain the same number of items. {dataset}.inter.json must map each user ID to its full chronological item sequence. {dataset}.item.json must map each internal item ID to its metadata, including the title and description used by the text encoder.

The Tokenization stage generates the paired semantic IDs for all nine datasets and the text-only/image-only semantic IDs for the three target datasets. Do not prepare these files manually:

{dataset}.index_umagr.json
{dataset}.index_umagr_text.json
{dataset}.index_umagr_image.json

For the missing-modality workflow, also place the fixed evaluation manifests under data/missing_manifests:

data/missing_manifests/
└── {arts,games,instruments}_{ot,oi,random}_r{25,50,75,100}_seed2026.manifest.json

These manifests are fixed protocol inputs, not randomly regenerated by this repository. Each manifest must contain dataset, missing_mode, missing_ratio, seed, assignments, selected_count, total_items, and assignment_counts, with assignments covering the complete evaluation item catalog. For ot, oi, and random, missing_mode must be text_only, image_only, and random_missing, respectively; assignment routes must use full, text, or image.

Fully observed

Set the shared paths once:

export GPU=0
export DATA_ROOT=/path/to/data
export OUT_BASE="$PWD/outputs"

1. Tokenization

Train the multimodal tokenizer and generate paired, text-only, and image-only semantic IDs:

RUN_TOKENIZER=1 RUN_INDEX=1 \
RUN_PRETRAIN=0 RUN_FINETUNE=0 RUN_EVAL=0 \
bash run_fully_observed.sh

Paired consensus uses (reliability + 1e-6) / (text_reliability + image_reliability + 2e-6).

2. Pre-training

RUN_TOKENIZER=0 RUN_INDEX=0 \
RUN_PRETRAIN=1 RUN_FINETUNE=0 RUN_EVAL=0 \
bash run_fully_observed.sh

3. Fine-tuning

Fine-tune and evaluate Arts, Games, and Instruments:

RUN_TOKENIZER=0 RUN_INDEX=0 RUN_PRETRAIN=0 \
RUN_FINETUNE=1 RUN_EVAL=1 \
bash run_fully_observed.sh

To run all three stages in one command:

bash run_fully_observed.sh

The final command writes:

outputs/fully_observed/unpairgr_fully_observed_results.csv

Missing modality

Generate the paired, text-only, and image-only semantic IDs with the fully observed workflow first. Then place the fixed evaluation manifests under data/missing_manifests and run:

GPU=0 \
DATA_ROOT=/path/to/data \
MANIFEST_ROOT=/path/to/data/missing_manifests \
FULLY_OBSERVED_ROOT="$PWD/outputs/fully_observed" \
OUT_BASE="$PWD/outputs/missing_modality" \
bash run_missing_modality.sh

This evaluates text-only (OT, paper TO), image-only (OI, paper IO), and random missingness at 25/50/75/100%, and writes:

outputs/missing_modality/unpairgr_missing_results.csv

Citation

If you find this work useful, please cite:

@article{shen2026unpairgr,
  title   = {Unpaired Modality-Agnostic Generative Recommendation},
  author  = {Shen, Weihao and Chen, Wei and Zhang, Fuwei and Yuan, Meng and
             Lan, Yuqin and Liu, Guojun and Hua, Qingsong and Lin, Wei and
             Zhuang, Fuzhen},
  journal = {arXiv preprint arXiv:2608.02477},
  year    = {2026}
}