Skip to content

fix: add dataclass-aware YAML field type validation at parse time#192

Merged
Ruari-Phipps merged 4 commits into
mainfrom
ruari/fix/validation_error_wrong_type
Jun 17, 2026
Merged

fix: add dataclass-aware YAML field type validation at parse time#192
Ruari-Phipps merged 4 commits into
mainfrom
ruari/fix/validation_error_wrong_type

Conversation

@Ruari-Phipps

@Ruari-Phipps Ruari-Phipps commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds automatic type validation for YAML resource fields at parse time, catching the colon-space footgun where unquoted strings silently parse as dicts.

Motivation

YAML values with unquoted mid-sentence colons (e.g., the outcome: either a time slot or...) silently parse as dicts instead of strings. This causes opaque TypeError: bad argument type for built-in operation from protobuf C code at push time — with no hint about which field or file caused it. A confirmed case in prompt_assertions burned ~7 push attempts before the user found the cause.

Supersedes #191 — moves the detection from project.py's generic error handler down to the YAML resource layer with field-level precision.

Changes

  • Add check_yaml_field_types() in resource.py using dataclasses.fields() + typing.get_type_hints() to introspect declared types and validate str / list[str] fields aren't dicts
  • Recurses into nested dataclass fields (catches e.g. TestCaseAssertion.prompts inside TestCase)
  • Hook into YamlResource.read_local_resource — all YAML resources get the check automatically
  • Hook into FlowStep and Pronunciation overriding read_local_resource methods
  • 6 unit tests covering str→dict, list[str]→dict, nested recursion, valid passthrough, Optional[str] None, and error message content

Test strategy

  • Added/updated unit tests
  • Manual CLI testing (poly <command>)
  • Tested against a live Agent Studio project
  • N/A (docs, config, or trivial change)

Checklist

  • ruff check . and ruff format --check . pass
  • pytest passes
  • No breaking changes to the poly CLI interface (or migration path documented)
  • Commit messages follow conventional commits

Screenshots / Logs

Example error output for the confirmed bug (test case with unquoted colon in prompt assertion):

ValueError: Error reading resource test_booking at test_suite/test_booking.yaml:
'assertions.prompts[1]' should be a string but got a dict.

🤖 Generated with Claude Code

YAML values with unquoted colons silently parse as dicts instead of
strings, causing opaque TypeError from protobuf C code at push time.
Uses dataclasses.fields() + typing.get_type_hints() to automatically
validate str and list[str] fields in YamlResource.read_local_resource,
with recursion into nested dataclass fields.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Ruari-Phipps Ruari-Phipps requested a review from a team June 15, 2026 16:18
@github-actions

This comment has been minimized.

A list[str] field receiving a dict (e.g., prompt_assertions parsed as a
mapping) would iterate dict keys as strings, silently passing the check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

This comment has been minimized.

- Check all scalar types (str, int, float, bool) not just str
- Validate dict[K, V] keys and values when K/V are scalar types
- Accept int where float is expected (YAML parses 500 as int)
- Reject bool where int/float is expected (yes/no auto-cast)
- Validate list[str] field is actually a list, not a dict
- Remove verbose YAML hint from error messages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report

Base (main) PR Change
70.0% 70.2% +0.1% ✅

Changed file coverage

File Coverage Change
poly/resources/pronunciation.py 79.7% +0.3% ✅
poly/resources/resource.py 82.7% +0.1% ✅
poly/resources/flows.py 86.1% +0.1% ✅
poly/resources/resource_utils.py 86.4% +1.5% ✅

@Ruari-Phipps Ruari-Phipps merged commit c871f34 into main Jun 17, 2026
5 checks passed
@Ruari-Phipps Ruari-Phipps deleted the ruari/fix/validation_error_wrong_type branch June 17, 2026 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants