Sub-issue of #376. The founding "how do we test its effectiveness?" acceptance. Owned by #376 (not shared with #377). Depends on #459 and #460.
Scope
- Promptfoo harness with a corpus of natural-language asks (e.g. "access-controlled treasury with a timelock and a rate limiter") → expected recipe id(s).
- Deterministic scoring vs
examples/ as ground truth:
tool-call-f1 — correct recipe(s) selected;
sui move build on the scaffolded package;
r.mvr deps resolve;
sui move test passes;
- zero anti-pattern hits.
- Run pre-release / on schedule (needs Sui CLI + model calls).
Acceptance: harness runs against the live server and reports a pass/fail score per ask; seed corpus authored.
Sub-issue of #376. The founding "how do we test its effectiveness?" acceptance. Owned by #376 (not shared with #377). Depends on #459 and #460.
Scope
examples/as ground truth:tool-call-f1— correct recipe(s) selected;sui move buildon the scaffolded package;r.mvrdeps resolve;sui move testpasses;Acceptance: harness runs against the live server and reports a pass/fail score per ask; seed corpus authored.