From PR #346 review (2026-07-02). Grouped low/medium cleanup items.
bfcl_v4_scorer.py:204 — live_relevance inverted semantics published. It takes the _score_ast path with ground_truth="[]", awarding credit for NOT calling; BFCL relevance is the opposite (correct when the model DOES call). It's excluded from category aggregates via _UNSCORED_SUBSETS, but the backwards value is still emitted under results.json "unscored_subsets". Either score with relevance semantics (1.0 if has_calls) or drop it from output.compliance/checker.py:187 — silent PASS when no accuracy metric applies. If no _ACCURACY_METRIC_KEYS key intersects the model's golden_accuracy, check_accuracy returns zero accuracy:* checks and ComplianceReport.passed reports PASS on config-lock alone. "No applicable accuracy metric" should be an explicit FAIL/warning.load_generator/session.py:380 — stop_current_phase doesn't set _drain_event. If the max_duration_ms cap fires while the perf phase is already in its drain wait, the callback is a no-op; with drain.performance_timeout_s: null a hung request keeps the run alive indefinitely despite the cap.metrics/results_plots.py:358 — one-directional length guard. counts/buckets are extracted independently with no length validation; a shorter hist_counts makes axes[1].bar(...) raise a shape-mismatch ValueError, and generate_plots doesn't catch it, aborting plotting for the whole report dir. Validate both arrays to a common length.dataset_manager/predefined/bfcl_v4/multi_turn.py:120 — dead/duplicated plumbing. get_tools_for_turn/excluded_function unused; MULTI_TURN_SUBSETS duplicated with the package __init__.py; DEFAULT_MAX_STEPS_PER_TURN=25 duplicated in bfcl_v4_execution.py:47 and bfcl_v4_multi_turn_runner.py:40. Risk of silent drift.openai/openai_msgspec_adapter.py:88 — dead guard. ModelParams.max_new_tokens is non-optional with a default, so is not None is always true; the omission path is unreachable. Make the field int | None or drop the guard.
Deferred out of PR #346.
From PR #346 review (2026-07-02). Grouped low/medium cleanup items.
bfcl_v4_scorer.py:204—live_relevanceinverted semantics published. It takes the_score_astpath withground_truth="[]", awarding credit for NOT calling; BFCL relevance is the opposite (correct when the model DOES call). It's excluded from category aggregates via_UNSCORED_SUBSETS, but the backwards value is still emitted underresults.json"unscored_subsets". Either score with relevance semantics (1.0 if has_calls) or drop it from output.compliance/checker.py:187— silent PASS when no accuracy metric applies. If no_ACCURACY_METRIC_KEYSkey intersects the model'sgolden_accuracy,check_accuracyreturns zeroaccuracy:*checks andComplianceReport.passedreports PASS on config-lock alone. "No applicable accuracy metric" should be an explicit FAIL/warning.load_generator/session.py:380—stop_current_phasedoesn't set_drain_event. If themax_duration_mscap fires while the perf phase is already in its drain wait, the callback is a no-op; withdrain.performance_timeout_s: nulla hung request keeps the run alive indefinitely despite the cap.metrics/results_plots.py:358— one-directional length guard.counts/bucketsare extracted independently with no length validation; a shorterhist_countsmakesaxes[1].bar(...)raise a shape-mismatchValueError, andgenerate_plotsdoesn't catch it, aborting plotting for the whole report dir. Validate both arrays to a common length.dataset_manager/predefined/bfcl_v4/multi_turn.py:120— dead/duplicated plumbing.get_tools_for_turn/excluded_functionunused;MULTI_TURN_SUBSETSduplicated with the package__init__.py;DEFAULT_MAX_STEPS_PER_TURN=25duplicated inbfcl_v4_execution.py:47andbfcl_v4_multi_turn_runner.py:40. Risk of silent drift.openai/openai_msgspec_adapter.py:88— dead guard.ModelParams.max_new_tokensis non-optional with a default, sois not Noneis always true; the omission path is unreachable. Make the fieldint | Noneor drop the guard.Deferred out of PR #346.