feat(cli): config-file-driven rllm train / rllm eval (phases 1–2)

[jeffreysijuntan]

What

Adds self-contained config-file mode to both rllm train and rllm eval: one TOML/YAML file specifies the whole run, launched as rllm train run.toml [key=value ...] / rllm eval run.toml [key=value ...]. Implements phases 1–2 of design/config-file-driven-train-eval.md (included in this PR).

A cookbook run — e.g. terminal-rl's train.py + train_fireworks_a35b.sh (~220 lines) — collapses to one ~40-line run.toml, and the same file drives eval.

Schema

backend = "fireworks"                     # tinker | fireworks | verl

[run.agent]                               # run definition (was train.py code)
name = "terminus2"                        # catalog name | module:Class import path
[run.agent.args]
max_turns = 75
[run.dataset]
train = "tb-opus-pass"
val   = "terminal-bench@2.0"
[run.sandbox]
backend = "modal"
[run.env]                                 # exported before the run
RLLM_HARNESS_RUN_TIMEOUT_S = "2400"

[model]                                   # everything else mirrors the config tree 1:1
name = "accounts/fireworks/models/qwen3p5-35b-a3b"
[training]
group_size = 16
[rllm.async_training]
enable = true

[eval]                                    # eval-only knobs (rllm eval)
split = "default"
concurrency = 64
attempts = 1
model = "accounts/fireworks/models/qwen3p5-35b-a3b"   # or base_url / rllm setup

Components

• rllm/config/run_config.py — parse .toml/.yaml → (DictConfig, RunSpec). Composes the full trainer config for the file's backend via Hydra's compose API (handles the @package _global_ split, ${...} interpolations, verl's defaults: /ppo_trainer), merges config-tree sections, mirrors [data]→[rllm.data], resolves extends, applies dotlist overrides.
• rllm/cli/_run_resolve.py — resolve a RunSpec into agent_flow/evaluator/datasets (+ entrypoint escape hatch for train), reusing load_agent/load_evaluator/DatasetRegistry/BenchmarkLoader.
• rllm/cli/train.py — config-file positional → config driver; --dry-run prints the composed config + resolved run.
• rllm/cli/eval.py — config-file positional → config driver reading [run] + [eval]; --dry-run.
• rllm/eval/agent_loader.py — load_agent(name, agent_args=...) threads agent_args as constructor kwargs (class) or via configure() (instance). None = unchanged.

Consolidation (bundled)

• Single eval-resolution path: config-eval funnels into the same _run_eval core as flag-eval (no duplicated dataset/agent/evaluator/materialization logic); _run_eval gained an agent_args param.
• Shared model-source/proxy helper: extracted _resolve_eval_model_source from eval_cmd; flag mode and config mode both use it (one proxy-lifecycle implementation). Flag-path user-facing strings are byte-identical.
• agent_args handling centralized in load_agent (used by train, eval, and the resolver).

Testing

• tests/config/test_run_config.py (17) — parse, backend compose (tinker/fireworks), [data] mirror + precedence, extends, dotlist overrides, RunSpec (nested + scalar shorthands), export_env, errors.
• tests/eval/test_agent_loader.py (+3) — agent_args as constructor kwargs (import-path + catalog-name), None unchanged.
• tests/cli/test_eval_command.py (+4) — eval config dry-run, missing-dataset error, _resolve_eval_model_source direct-mode (+ requires-model).
• Full tests/cli + tests/config + tests/eval/test_agent_loader.py = 133 passing; ruff clean.
• Smoke-tested --dry-run for both train and eval on a fireworks config: composes config, resolves terminus2 + datasets, applies dotlist overrides, renders headers.

Deliberately deferred (follow-ups)

• verl backend file-mode verified only via config compose here; needs a run on a box with verl installed (its ppo_trainer searchpath).
• entrypoint-for-eval — declarative only for now; fails with a clear message.
• _run_train → shared resolver / build_train_config → merge_backend_config — the flag paths still have their own compose/resolution. Left as a separate refactor to avoid regressing the working flag training path in this PR.
• Cross-backend canonicalization — promoting model/optimizer/LoRA/group-size into rllm.* + extending each _SHARED_KEYS so flipping backend carries all common config (design §"Backend selection & cross-backend portability").
• Reference cookbook migration (terminal-rl → run.toml) + rllm-docs page.

🤖 Generated with Claude Code

[jeffreysijuntan]

Superseded by the terminal-rl-based PR (this branch targeted main; terminal-rl has diverged train.py/eval.py).