Skip to content

fix(cookbooks): normalize LaTeX and markdown formatting in Fireworks … - #659

Open
shobhitagnihotri69 wants to merge 1 commit into
hud-evals:mainfrom
shobhitagnihotri69:fix/grader-normalize-llm-formatting
Open

shobhitagnihotri69 wants to merge 1 commit into
hud-evals:mainfrom
shobhitagnihotri69:fix/grader-normalize-llm-formatting

Conversation

@shobhitagnihotri69

@shobhitagnihotri69 shobhitagnihotri69 commented Sep 10, 2026

Copy link
Copy Markdown

Summary

The arithmetic task in cookbooks/fireworks-rl-training checks the final integer on the last non-empty line. However, reasoning models (such as Qwen 2.5 and DeepSeek-R1) standardly format their final answers wrapped in LaTeX boxing (\boxed{...}, $\boxed{...}$), markdown bold (**...**), or inline code tags.

Previously, re.fullmatch(r"[+-]?(?:\d+|\d{1,3}(?:,\d{3})+)", final) on the final line rejected these common wrappers, assigning reward = 0.0 to mathematically correct rollouts and corrupting the RL training reward signal.

Changes

  • Unwraps \boxed{...} and strips enclosing markdown/LaTeX delimiters ($, *, `, .) from the final answer line prior to integer matching.
  • Restores expected reward 1.0 for \boxed{...} and adds test coverage for $\boxed{...}$, **...**, and `...` in test_train.py.
  • Syncs the cookbook documentation snippet in docs/v6/cookbooks/fireworks-rl-training.mdx.

Verification

  • Ran test suite via uv run pytest test_train.py -k test_grades_integer_on_last_nonempty_line (22 passed, 100%).
  • Passed ruff check and ruff format.

Note

Low Risk
Cookbook-only grader and test/doc updates; no auth, infra, or broader SDK behavior changes.

Overview
The Fireworks RL cookbook’s grade_final_integer grader now normalizes common answer wrappers on the last line before parsing the integer, so correct math isn’t scored as wrong when models use typical reasoning-model formatting.

After taking the last non-empty line, it unwraps \boxed{...} and strips enclosing $, *, backticks, whitespace, and trailing periods, then applies the existing comma-aware integer regex. Tests now expect full reward for \boxed{...}, $\boxed{...}$, **...**, and `...` answers. The v6 cookbook doc snippet matches env.py.

Reviewed by Cursor Bugbot for commit 3bce83c. Bugbot is set up for automated code reviews on this repo. Configure here.

…arithmetic grader

LLMs frequently format final integer answers with LaTeX boxing (\boxed{...}, $\boxed{...}$) or markdown bold/code tags on the final line.

Previously, re.fullmatch rejected this formatting, causing correct model rollouts to receive reward 0.0 and corrupting RL training signals.

This unwraps \boxed{...} and strips markdown/LaTeX delimiters from the final answer line, updates the test expectations, adds tests for formatted answers, and syncs the cookbook documentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants