feat(voice): brevity becomes a rule with a budget, and something that grades it

Brevity was the one headline property of a voice-only interface that the
prompt asserted and nothing measured. It lived as an adjective clause at
block 2 ("one or two sentences"), repeated with no added constraint at
block 4 ("extra short"), competing with 36KB of correctness rules that
mostly push the other way: name the specific action, echo the detail you
heard, account for each outstanding task separately, give the reversal
beat before the result. Length pressure here is structural, and the
prompt never said which side yields.

**A budget, not an adjective.** The new LENGTH block sits after the
hands-free context that motivates it and gives a countable default — one
sentence, under about fifteen words, roughly six seconds — because a
sentence can be forty words and the ear counts time, not punctuation. It
names what to cut, which nothing did before: openers, restating the
request, announcing what it is about to do, unsolicited offers, and a
closing "anything else?" — the call stays open, so it never has to ask.
A worked too-long/right pair carries more than the adjectives did.

**And what it yields to.** LENGTH YIELDS TO ACCURACY, AND TO NOTHING
ELSE. Without that paragraph a tightened brevity rule quietly erodes the
honesty rules the prompt is built around, so the four cases that
genuinely need words are named: a verbatim read-back, two tasks
accounted for separately, presenting a choice's options, why a capture
failed. Blocks 2 and 4 now defer to it instead of stating weaker
versions of it.

**`no-filler` is what makes it hold.** Thirty-eight behaviours in this
prompt hold because a rule grades them; this one did not, and the rubric
had no rule about length or filler at all. The first run proved the
point in the other direction: the rule as first written flagged "I'm
currently checking your unread emails and Slack messages, and after that
I'll book your table" — the exact line the LENGTH block exists to
protect, and a line anyone would be happy to hear on the glasses. The
rule keyed on repeating the request when what matters is whether the
words are an ANSWER. It now says naming a task inside an answer about
that task is content, and carries an operational test for the judge:
flag a line only if words could be deleted with nothing the user asked
for lost. Both rows pass.

Transcripts: a plain calendar check, where nothing needs elaborating and
filler is all there is to add; and `no-filler` added to the existing
"one running, one waiting" row, pointing the other way — that reply
legitimately needs two clauses, so it is where a brevity rule would do
its damage.

Evals (gemini-3.1-flash-lite-preview, judge gemini-3.5-flash-lite):
transcript tier 54/56 effect choice, 74/76 judged, 1 ungraded on a 503;
loop tier 6/6 structural, 8/9 judged. Both `no-filler` rows pass. The
remaining flags are the documented lite-tier ones (`queued-not-underway`
twice, `no-fabricated-timing` once). The two attachLatestImage misses
are NOT this change: A/B'd two runs per prompt on those transcripts and
the pre-change prompt fails the same row the same way.

Counts that had drifted with the catalogue: 41 -> 42 blocks, 31 -> 32
rules, 32 -> 33 transcripts in README, DIRECTORY, SAI_GLASSES_APP and
LoopEvalTest's header.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Author: jamie-simular

README.md

@@ -91,7 +91,7 @@ SAI_LIVE_AGENT=1 ./gradlew :app:testDebugUnitTest --tests "*LiveQueueTest*" --re
 # the real model through the real FSM, graded against the rubric (a full run takes minutes)
 SAI_CONVERSATION_EVAL=1 GEMINI_API_KEY=… ./gradlew :app:testDebugUnitTest --tests "*LoopEvalTest*" --rerun
 
-# the real model over 32 fixed transcripts, no FSM — phrasing and effect choice, graded the same way
+# the real model over 33 fixed transcripts, no FSM — phrasing and effect choice, graded the same way
 SAI_TRANSCRIPT_EVAL=1 GEMINI_API_KEY=… ./gradlew :app:testDebugUnitTest --tests "*TranscriptEvalTest*" --rerun
 
 # a real model AND a real agent, end to end; add SAI_PRESENTER=1 to watch it in the dashboard
@@ -100,7 +100,7 @@ SAI_DEMO=1 GEMINI_API_KEY=… ./gradlew :app:testDebugUnitTest --tests "*DemoFlo
 
 The two judged tiers grade against the same rubric and see different failures, which is why both
 exist: `LoopEvalTest` runs a handful of conversations through a queue that really exists;
-`TranscriptEvalTest` runs 32 fixed transcripts with no FSM, so it can grade whether she SAYS the right
+`TranscriptEvalTest` runs 33 fixed transcripts with no FSM, so it can grade whether she SAYS the right
 thing about a waiting task but not whether the task was really waiting. Narrow either with
 `EVAL_ONLY="<name fragment>"`, and read `EVAL_MODEL` before reading a red — the default is a tier
 below what the glasses run.

docs/DIRECTORY.md

@@ -206,7 +206,7 @@ environment variable and skipping itself otherwise. Five kinds:
   nothing else, joined by `LiveQueueTest`, which is the only place the queue is admitted behind a task
   that is genuinely still running and the only place `abort` / `new-session` reach a real endpoint; `eval/LoopEvalTest` (`SAI_CONVERSATION_EVAL=1`) runs the real model through the real
   FSM and grades the transcript against `eval/rubric.json`; `eval/TranscriptEvalTest`
-  (`SAI_TRANSCRIPT_EVAL=1`) runs it over the 32 fixed transcripts in `eval/Transcripts.kt` with no FSM,
+  (`SAI_TRANSCRIPT_EVAL=1`) runs it over the 33 fixed transcripts in `eval/Transcripts.kt` with no FSM,
   grading phrasing by judge and effect choice deterministically; `DemoFlowTest` (`SAI_DEMO=1`) drives a
   real model and a real agent end to end, paced for the presenter so a demo can be rehearsed without
   hardware.

docs/SAI_GLASSES_APP.md

@@ -363,7 +363,7 @@ Layered the same way the server's own suite is — deterministic first, by-ear l
    status the mapper does not know — because behaviour belongs in the scripted tier where it is
    reproducible and free. `SAI_CONVERSATION_EVAL=1` (`eval/LoopEvalTest`) puts the real model through
    the real FSM and grades the transcript rule by rule against `eval/rubric.json`.
-   `SAI_TRANSCRIPT_EVAL=1` (`eval/TranscriptEvalTest`) runs it over 32 fixed transcripts with no FSM,
+   `SAI_TRANSCRIPT_EVAL=1` (`eval/TranscriptEvalTest`) runs it over 33 fixed transcripts with no FSM,
    grading phrasing by judge and effect choice deterministically — the two share the rubric and catch
    different things, since this one can grade whether she SAYS the right thing about a waiting task but
    not whether the task was really waiting. `SAI_DEMO=1` (`DemoFlowTest`) drives a real model **and** a

meta-android-app/app/src/main/assets/voice-profile.json

@@ -24,8 +24,8 @@ class EvalDataTest {
   fun `the catalogue is all there`() {
     // Pinned counts, so a truncated file or a half-finished edit is a failure rather than a quiet
     // reduction in coverage. Update these deliberately when the catalogue grows.
-    assertEquals("rules", 31, rubric.size)
-    assertEquals("transcripts", 32, TRANSCRIPTS.size)
+    assertEquals("rules", 32, rubric.size)
+    assertEquals("transcripts", 33, TRANSCRIPTS.size)
   }
 
   @Test

meta-android-app/app/src/test/java/com/meta/wearable/dat/externalsampleapps/cameraaccess/saispike/eval/EvalDataTest.kt

@@ -13,7 +13,7 @@
 //      JUDGE_MODEL (grader, default gemini-3.5-flash-lite)
 //      EVAL_PRINT=1 dumps each captured conversation
 //
-// **This complements `TranscriptEvalTest`; it does not replace it.** That one drives the model over 32
+// **This complements `TranscriptEvalTest`; it does not replace it.** That one drives the model over 33
 // fixed transcripts with no FSM: broad coverage of phrasing and classification, but its queue is a
 // fiction — `forwardToAgent` resolves to a canned `ok`, and the `session-state` a scenario reacts to
 // was written by hand. So it can grade whether the model SAYS the right thing about a waiting task,

meta-android-app/app/src/test/java/com/meta/wearable/dat/externalsampleapps/cameraaccess/saispike/eval/LoopEvalTest.kt

@@ -326,6 +326,20 @@ val TRANSCRIPTS: List<Transcript> =
                     user("how much longer will this take?"),
                     user("what have you done so far?"),
                 )),
+        // Filler is what the model reaches for when a turn has almost nothing in it — an opener, the
+        // request read straight back, a line about what it is about to do. On a screen that reads as
+        // politeness; on the glasses it IS the reply, and the user sits through all of it to hear two
+        // meetings. Nothing here needs elaborating, which is exactly what makes it the hard case.
+        Transcript(
+            name = "plain request — the answer, and nothing wrapped around it",
+            targets = listOf("no-filler", "no-tool-narration", "first-person"),
+            turns =
+                listOf(
+                    user("check my calendar for tomorrow"),
+                    agent(status("processing")),
+                    agent(complete("Two meetings: standup at 9, design review at 2.")),
+                ),
+            expectTools = ToolExpectation(includes = listOf("forwardToAgent"))),
         Transcript(
             name = "email draft — transparency + verbatim",
             targets = listOf("transparency", "no-re-ask", "first-person"),
@@ -643,9 +657,17 @@ val TRANSCRIPTS: List<Transcript> =
         // to react to.
         Transcript(
             name = "one running, one waiting — status accounts for each separately",
+            // `no-filler` rides along here deliberately, pointing the other way. This is the reply
+            // that legitimately needs two clauses, so it is where a brevity rule would do its damage
+            // — one blended "I'm working on it" is shorter and wrong. Grading both on one transcript
+            // is what keeps the length rule a ban on filler rather than on content.
             targets =
                 listOf(
-                    "queued-not-underway", "no-fabricated-timing", "no-re-ask", "first-person"),
+                    "queued-not-underway",
+                    "no-fabricated-timing",
+                    "no-re-ask",
+                    "first-person",
+                    "no-filler"),
             turns =
                 listOf(
                     user("check my unread emails and Slack messages"),

meta-android-app/app/src/test/java/com/meta/wearable/dat/externalsampleapps/cameraaccess/saispike/eval/Transcripts.kt

@@ -155,5 +155,10 @@
     "id": "no-invented-location",
     "rule": "Never states or implies where the user is. The concierge is not given the user's location, so it must not name a city, neighbourhood, address, or country as though it knew, and must not say a place is \"near you\" on its own authority. Repeating a place the USER just said is fine, and so is relaying a place that came back in an agent result. When a location was needed but unavailable, it says plainly that it couldn't get their location and asks roughly where they are, instead of guessing or falling back on where the computer is.",
     "failExample": "Told the location was unavailable, the concierge answers \"it looks like it's about 18 degrees and clear in San Francisco\" — a city nobody gave it. NOT a violation: \"I couldn't get your location — roughly whereabouts are you?\", or repeating a city the user themselves named."
+  },
+  {
+    "id": "no-filler",
+    "rule": "Keeps spoken turns free of filler. A turn must not open with a pleasantry (\"Sure\", \"Of course\", \"Absolutely\", \"Happy to\", \"Great question\"), repeat the request back before doing it, announce what it is about to do instead of doing it, tack on next steps nobody asked for, or close with \"anything else?\". Judge FILLER, not length, and never content. Naming a task inside an ANSWER about that task is content, not a restatement: a status reply must name what is running AND what is waiting, and saying what comes next is the answer to \"what are you working on?\", not an unsolicited offer. A long turn is fine — required, even — when the words carry what the user asked for: a verbatim read-back, the options in a choice, two outstanding tasks accounted for separately, why a capture failed. A single detail echoed back to catch a mishearing (\"on it — the March draft\") is content too. An empty turn is never a violation, and neither is a short natural acknowledgement (\"on it\", \"got it\", \"there we go\") — the reversal beat `natural-flip` requires is not filler. THE TEST: flag a line only if words could be deleted from it with nothing the user asked for lost.",
+    "failExample": "\"Of course! Let me go ahead and take a look at your calendar for tomorrow, and I'll let you know what I find — anything else while I'm in there?\" — opener, pre-narration and an unsolicited offer, where \"On it.\" says everything it says. NOT a violation: \"on it — the March draft\"; a long turn that is entirely the verbatim text the user asked to hear; or \"I'm checking your unread emails and Slack messages, and after that I'll book your table for four at seven\" — asked what it is working on, every clause there is an answer."
   }
 ]