You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
compliance/checker.py is edge-agentic-specific (BFCL metric keys, qwen3.6-27b / mlperf-edge-current defaults, hardcoded attestation note) under a generic top-level compliance/ name.
Proposed:
Rename to compliance/edge_agentic.py (or move next to the edge ruleset).
Make ruleset_name/model_name required args instead of defaults.
Name the model's golden_accuracy keys to match the scorer's breakdown keys (overall_accuracy, normalized_single_turn_score) so the _ACCURACY_METRIC_KEYS mapping disappears.
Follow-up to PR #346 (thread #346 (comment)), flagged by @viraatc as fine to address in a follow-up.
compliance/checker.pyis edge-agentic-specific (BFCL metric keys,qwen3.6-27b/mlperf-edge-currentdefaults, hardcoded attestation note) under a generic top-levelcompliance/name.Proposed:
compliance/edge_agentic.py(or move next to the edge ruleset).ruleset_name/model_namerequired args instead of defaults.golden_accuracykeys to match the scorer's breakdown keys (overall_accuracy,normalized_single_turn_score) so the_ACCURACY_METRIC_KEYSmapping disappears.