Breaking & deprecation changes since v0.3

[eric-haibin-lin]

This is a change log specifically focused on breaking changes so that the community is aware.

class interfaces

SFTDataset interface

• interface change, no change in functionality
• [BREAKING] feat: support custom datasets for SFT trainer #832

RLHFDataset interface

• similar to SFTDataset, it now takes a config as input
• feat: support customized datasets for RayPPOTrainer #924

FSDPSFTTrainer:

• 3 arguments are now required: tokenizer, train_dataset, val_dataset https://github.com/volcengine/verl/pull/1282/files

config

If you come across usage such as +{config}={value} that throw an error:

These breakings are usually caused by consolidating a config term into the yaml file. Please try removing the + to fix such errors.

Here are the changed configs:

• trainer.val_before_train: [BREAKING config] fix: move val_before_train to config yaml. Using trainer.val_before_train instead of +trainer.val_before_train going forward #820
• (sft_trainer.yaml) data.{prompt,response}_dict_keys: [ci] feat: improve CI speed #1032

[HUGHNew]

config

(sft_trainer.yaml) change the default value to make them same as PPO (trainer.default_local_dir, trainer.default_hdfs_dir): #1451

[hongpeng-guo]

utils

verl.utils.reward_score._default_compute_score is deprecated, please use verl.utils.reward_score.default_compute_score instead. See #1397 and #1729

[eric-haibin-lin]

entropy_coeff now defaults to 0 #1770

[eric-haibin-lin]

Breaking changes since v0.4 recorded in #1902