Shared ideas: contract validation + self-healing across Python/Ruby ecosystems

[Correctover]

Hi! 👋

I came across ruby_llm-contract and I'm really impressed by the approach — validating LLM JSON outputs against contracts with automatic retry is exactly the right direction. We've been solving a similar problem in the Python ecosystem with Correctover and I think there's a lot of overlap (and potential learning) between our approaches.

What we've learned building Correctover

Correctover (https://correctover.com / https://pypi.org/project/correctover/) takes a broader approach than just JSON schema validation. Our CANON framework validates across 6 dimensions:

1. Contract — JSON schema / structure match
2. Acceptance — Semantic equivalence (embedding cosine similarity)
3. Non-functional — Latency budget, token limits
4. Observability — Trace metadata presence
5. Normative — Policy / compliance checks
6. (extensible) — Custom validators via plugin

A few things we've found that might be relevant to ruby_llm-contract:

1. Surgical regeneration (like your #31!)
Your proposal for surgical regeneration as a third retry strategy aligns perfectly with our "self-healing engine." We use a 3-level recovery:

• L1: Simple retry with exponential backoff
• L2: Provider degradation (switch to fallback with modified params)
• L3: Full failover to alternative provider

The key insight: not all failures need the same remedy. Schema violations need surgical regeneration; timeouts need backoff; auth errors need provider switching.

2. Stochastic verifier challenge
Embedding-based semantic checks are "depth 0" if the embedding model can be silently swapped. Our solution: pin the scorer by content address (model ID + weights digest) and store both texts in the evidence trace.

3. Flywheel learning
Our self-healing engine learns from failures — 88 flywheel rules auto-generated from 20,071 benchmark calls. The system gets smarter over time without manual intervention.

Benchmark comparison?

I'd love to see how ruby_llm-contract's validation + retry compares to Correctover's approach. Would you be open to a collaborative benchmark?

Potential collaboration

• Ruby ↔ Python: Different ecosystems, same problems — we could share test cases and failure taxonomies
• Cross-language contract format: A universal contract schema that works across languages
• Shared failure dataset: Our 20K-call benchmark covers 280+ fault patterns — happy to share the taxonomy

Would be great to chat about ideas. Thanks for the great work! 🚀

---

PyPI: https://pypi.org/project/correctover/
Website: https://correctover.com