Skip to content

FIX parse likert raw score via float before int#2133

Merged
romanlutz merged 2 commits into
microsoft:mainfrom
WatchTree-19:fix-likert-float-raw-score
Jul 8, 2026
Merged

FIX parse likert raw score via float before int#2133
romanlutz merged 2 commits into
microsoft:mainfrom
WatchTree-19:fix-likert-float-raw-score

Conversation

@WatchTree-19

Copy link
Copy Markdown
Contributor

problem

SelfAskLikertScorer raises ValueError when the judge returns a numeric likert value with a decimal point (e.g. "4.0"), which models commonly do for a "numeric value between 1 and 5".

in _score_async the raw value is parsed two different ways:

  • score value (works): float(unvalidated_score.raw_score_value)
  • metadata (crashes): int(unvalidated_score.raw_score_value)

raw_score_value is str(parsed_response["score_value"]), so a judge response of 4.0 becomes "4.0". float("4.0") is fine, but int("4.0") raises ValueError: invalid literal for int() with base 10: '4.0', so the whole score fails even though the float-scaled value computed correctly. SelfAskScaleScorer doesn't hit this because it never int()s the raw value.

fix

parse the metadata via float first, matching the score-value line:

score.score_metadata = {"likert_value": int(float(unvalidated_score.raw_score_value))}

test

added test_likert_scorer_accepts_float_string_score_value (judge returns score_value: "4.0"). fails on main (ValueError), passes with the fix, and asserts the likert_value metadata is 4. full tests/unit/score/test_self_ask_likert.py 34 pass; ruff check + ruff format clean.

found by reading the scorer, not from the tracker. happy to file an issue first if you'd prefer.

WatchTree-19 and others added 2 commits July 8, 2026 13:37
SelfAskLikertScorer crashed with ValueError when a judge returns a numeric
likert value like "4.0" (float() accepts it, int() does not). Parse the
likert_value metadata via float first, matching the score-value line. +test.

Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>
@romanlutz romanlutz changed the title fix(score): parse likert raw score via float before int FIX parse likert raw score via float before int Jul 8, 2026
@romanlutz romanlutz enabled auto-merge July 8, 2026 16:53
@romanlutz romanlutz added this pull request to the merge queue Jul 8, 2026
Merged via the queue into microsoft:main with commit 0bbeb71 Jul 8, 2026
52 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants