Skip to content

Commit 3305468

Browse files
jamie-simularclaude
andcommitted
feat(voice): the call wakes its own machine now, and says so
A hibernated machine used to come back only as a side effect of the first task, so that task absorbed the whole ~1min spin-up in silence and then produced a result. The three lines written for this — MACHINE_WAKING, MACHINE_AWAKE, MACHINE_WAKE_FAILED — have sat in Speech.kt unreferenced since the port, because the client had no way to detect or cause a wake. It does now (cloud-api #1635: POST /v1/agents/wake, plus status/canWake on GET /machines), so this wires them up. Decided behaviour: announce at the switch, and wake then. Not lazily on the next task, which is the gap being closed, and not eagerly for machines nobody switched to. The wake fires at first Live-ready and inside applyMachineSwitch, which both switch paths — the voice tool and the picker — already go through. WakePolicy holds the decisions, apart from the service, for the same reason GreetingGate and HangupPolicy do: each rule is a way of saying something untrue about a computer the wearer cannot see, and none of them is reachable from a JVM test while it lives in a foreground Service. - Branch on `startingUp`, not `waking`. A machine already mid-wake answers waking=false — correctly, nothing was dispatched — and is still owed the minute. Reading `waking` alone reintroduces the silent wake one field over. - Say nothing when canWake is false. That machine is asleep and staying that way, and the line promises about a minute. - The wake happens while muted; only the announcement is dropped, the same call AgentEventRouter makes for a notice. But the WATCH continues, gated on the machine rather than on whether we spoke — so unmuting mid-wake still hears "the computer's awake now", which is fresh news rather than a stale replay. - All three lines share one nudge kind, so a later one replaces an earlier one still held for the end of a turn. Otherwise the model gets "it's waking up" and "it's awake now" in one batch and reads out both. waking=true only ever meant DISPATCHED — vm-service is fire-and-forget and nothing polls it — so the three-minute timeout is this client's own and MACHINE_WAKE_FAILED is the honest end of it. Also adds check 8 to the on-device list, which had no switching check at all and, with it, finally has the ten it claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 3ca0534 commit 3305468

5 files changed

Lines changed: 502 additions & 4 deletions

File tree

docs/ON_DEVICE_CHECK.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ streams to a browser. DEBUG builds only, LAN only.
104104
> | The turn stream connects | `POST /v1/agents/message` answers with the turn's events. If that response is not read, nothing arrives from the agent at all — checks 2, 3 and 6 all go quiet together |
105105
> | Live opens with `?key=` | The URL form changed. The wrong one closes with `1007 api key not valid` |
106106
> | An idle call ends itself | The cost guard moved from the server to the phone. Leave a call silent for five minutes; it should hang up and say why |
107-
> | Waking a machine | The "it's waking / it's ready" lines now travel as `notice` over the stream |
107+
> | Waking a machine | **New on 2026-08-20 and never run on hardware.** The call now wakes its machine itself, at bind and at every switch, and narrates it: `MACHINE_WAKING``MACHINE_AWAKE`, or `MACHINE_WAKE_FAILED` after three minutes. Start a call on a **hibernated** machine — Sai should greet you and then say the computer is waking, verbatim, *before* you ask for anything. Silence there means `POST /v1/agents/wake` is not reaching the server (`wake:` lines in logcat say which); the old behaviour — a `notice` riding the first task's stream, a minute late — is the thing being replaced |
108108
> | Cancelling a queued task | Four endpoints deep, and the race answers are new — check 6 covers it |
109109
110110
## 1. The ten checks
@@ -256,6 +256,32 @@ mid-sentence.
256256
- **Then barge in while a task is running**, and let its completion land. Cutting Sai off must not
257257
cost you the result: it should still arrive after the exchange.
258258

259+
### 8. Switching machines
260+
261+
Mid-call, with more than one machine on the account:
262+
263+
> "Switch to my ⟨other machine⟩"
264+
265+
- **Expect:** a spoken "Switched to ⟨label⟩." and nothing more — then the **next** task runs on that
266+
machine, and `recallHistory` stops reaching the old one. Live audio stays up, so the conversation
267+
survives the switch.
268+
- **Exercises:** the `switchMachine` device tool, `MachineSwitcher.resolve` (exact, then containment
269+
either way — so "studio" finds "Studio Mac"), and `applyMachineSwitch`'s reconnect. `switched
270+
machine → ⟨label⟩` in logcat is the proof something was repointed; without it the next task still
271+
goes to the old machine.
272+
- **Watch for:** the reply's parenthetical read aloud. "Context update, not to be spoken aloud" in
273+
your ear is `RESELECT_NUDGE`'s failure in a new place. Switching from the **phone picker** instead
274+
must be silent — a button press carries no tool response, so the correction goes in as a nudge, and
275+
Sai narrating what you just pressed is the failure.
276+
- **Switch onto a hibernated machine** and it should wake, announced: "Switched to ⟨label⟩", then
277+
`MACHINE_WAKING` verbatim, then `MACHINE_AWAKE` on its own a minute or so later — all before you ask
278+
for anything. Both switch paths go through `applyMachineSwitch`, so the picker behaves the same.
279+
**The wake must stay silent when `canWake` is false** (a machine no provider can wake): that one is
280+
not coming back, and the line promises a minute. `WakePolicy` decides all of this and is JVM-tested;
281+
what the device is answering is whether the model reads a verbatim line as one, and whether
282+
"waking" and "awake now" ever arrive in the same breath — they share a nudge kind precisely so the
283+
later replaces the earlier, and hearing both is the failure.
284+
259285
### 9. Stopping work
260286

261287
The other half of the queue: check 6 is about work that waits, this is about work that dies.

meta-android-app/app/src/main/java/com/meta/wearable/dat/externalsampleapps/cameraaccess/saispike/CallService.kt

Lines changed: 149 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ import com.meta.wearable.dat.externalsampleapps.cameraaccess.BuildConfig
3939
import androidx.core.app.ServiceCompat
4040
import kotlinx.coroutines.CoroutineScope
4141
import kotlinx.coroutines.Dispatchers
42+
import kotlinx.coroutines.Job
4243
import kotlinx.coroutines.SupervisorJob
4344
import kotlinx.coroutines.cancel
4445
import kotlinx.coroutines.delay
@@ -106,6 +107,16 @@ class CallService : Service() {
106107
* are the only reason this is volatile at all.
107108
*/
108109
@Volatile private var saiMuted = false
110+
111+
/**
112+
* The bind-time wake has been asked for this call. One-shot, and deliberately NOT the greeting gate:
113+
* `greetOnFirstReady` returns early when muted, and a muted call still needs its machine woken —
114+
* mute suppresses the announcement, not the wake.
115+
*/
116+
@Volatile private var wakeRequested = false
117+
118+
/** The in-flight "is it awake yet" watcher, if any. A machine switch supersedes the previous one. */
119+
private var wakeWatch: Job? = null
109120
// Nudges withheld while muted (they'd be spoken into the void and lost). Replayed on unmute.
110121
private val heldNudges = HeldNudgeQueue(MAX_HELD_NUDGES)
111122
// Keepalive throttle: the last time we told the server a human is still present (see maybeKeepalive).
@@ -228,6 +239,9 @@ class CallService : Service() {
228239
audioPaused = false
229240
ending = false
230241
greetingGate.reset()
242+
wakeRequested = false
243+
wakeWatch?.cancel()
244+
wakeWatch = null
231245
activityLog.reset()
232246

233247
ServiceCompat.startForeground(
@@ -351,7 +365,17 @@ class CallService : Service() {
351365
// signal, and it is the only one that distinguishes a live call from an open mic.
352366
onUsage = { p, r, _ -> concierge?.onUsage(p, r) },
353367
onLog = { log(it) },
354-
onReady = { greetOnFirstReady() },
368+
onReady = {
369+
greetOnFirstReady()
370+
// AFTER the greeting, so the opening turn is the first thing heard rather than a status
371+
// line, and only once per call. Waiting for the Live session rather than firing at
372+
// startCall costs a second or two against a ~60s spin-up and buys both the ordering and
373+
// not waking a machine for a call that never connected.
374+
if (!wakeRequested) {
375+
wakeRequested = true
376+
wakeMachine(currentMachineId)
377+
}
378+
},
355379
onClosed = { if (callActive && !audioPaused && !ending) scheduleLiveReconnect() },
356380
)
357381
live = client
@@ -680,6 +704,9 @@ class CallService : Service() {
680704
hangupAt = 0L
681705
hangupJob?.cancel()
682706
hangupJob = null
707+
// Nothing to narrate to: the watcher's only output is speech on a call that no longer exists.
708+
wakeWatch?.cancel()
709+
wakeWatch = null
683710
// Drop the clipboard with the call. Held metadata outlives the WS otherwise, and the next call
684711
// would open with a photo from the last one silently eligible to attach — a stale picture riding
685712
// an unrelated request is the exact bug the hold-until-asked design exists to prevent.
@@ -1029,6 +1056,117 @@ class CallService : Service() {
10291056
if (notifyModel) {
10301057
live?.injectNudge("machine-switch", MachineSwitcher.contextNudge(match.label))
10311058
}
1059+
// Both switch paths land here — the voice tool and the picker — so the wake is wired once. It
1060+
// follows the context nudge for the same reason the bind-time one follows the greeting: the
1061+
// switch is what the user asked about, and the machine's state is the footnote.
1062+
wakeMachine(match.machineId)
1063+
}
1064+
}
1065+
1066+
/**
1067+
* Bring [machineId] up if it is asleep, and say so if it is.
1068+
*
1069+
* Decided behaviour: **announce at the switch, and wake then.** Not lazily on the first task — that
1070+
* is what this exists to fix. Until this path existed, the wake news could only ride a turn stream
1071+
* (`data-status` → `AgentEvent.Notice`), which is the response body of `POST /message`, so the first
1072+
* task of a call absorbed the whole ~1min spin-up in silence and then produced a result. Waking here
1073+
* moves the spin-up under the conversation instead.
1074+
*
1075+
* Three rules worth not undoing:
1076+
*
1077+
* 1. **Branch on `startingUp`, not `waking`.** A machine already mid-wake answers `waking = false` —
1078+
* correctly, nothing was dispatched — and the user is still owed the minute. `startingUp` is also
1079+
* false for a hibernated machine that cannot be woken, which is the case where saying the line
1080+
* would be a lie: `MACHINE_WAKING` promises about a minute, and that machine is not coming back.
1081+
* 2. **The wake happens even while muted; only the announcement is suppressed.** These three lines
1082+
* are true for about a minute and then describe a world that has moved on, so they are dropped
1083+
* rather than held — the same call `AgentEventRouter` makes for a `notice`. Replayed on unmute
1084+
* they would announce a wake that finished minutes ago.
1085+
* 3. **All three lines share one nudge kind**, so a later one REPLACES an earlier one still held for
1086+
* the end of a turn. Without that the model is handed "it's waking up" and "it's awake now" in one
1087+
* batch and reads out both — the contradiction `VoiceChannel.say`'s `supersedes` exists to stop.
1088+
*
1089+
* `waking = true` only means the wake was DISPATCHED: `ensureVmAwake` fires at vm-service
1090+
* fire-and-forget and never polls, so the timeout below is this client's own and
1091+
* [MACHINE_WAKE_FAILED] is the honest end of it.
1092+
*/
1093+
private fun wakeMachine(machineId: String) {
1094+
val p = params ?: return
1095+
// A switch retires the previous machine's watcher: whether the one we just left ever came up is no
1096+
// longer anything to tell the user about.
1097+
wakeWatch?.cancel()
1098+
wakeWatch =
1099+
scope.launch {
1100+
val token = SaiAuth.idToken() ?: p.token
1101+
val outcome =
1102+
try {
1103+
ConciergeClient.wakeMachine(p.baseUrl, token, machineId)
1104+
} catch (e: Exception) {
1105+
// Best-effort by design. A machine that cannot be woken is not a reason to fail a
1106+
// call — the first task will wake it the old way, a minute late.
1107+
log("wake: could not reach the wake endpoint — ${e.message}")
1108+
return@launch
1109+
}
1110+
1111+
val opening =
1112+
WakePolicy.onWakeRequested(
1113+
startingUp = outcome.startingUp,
1114+
muted = saiMuted,
1115+
audible = live != null,
1116+
status = outcome.status,
1117+
canWake = outcome.canWake,
1118+
dispatched = outcome.waking,
1119+
)
1120+
say(opening)
1121+
1122+
// Gated on the MACHINE, not on whether we spoke. Silence because nothing is coming up is the
1123+
// end of it; silence because we were muted is not — the wake is real and still in flight, and
1124+
// the mute may be over by the time it lands. `onWatchEnded` re-reads `saiMuted` at that
1125+
// moment, so unmuting mid-wake still hears "the computer's awake now", which is fresh news
1126+
// rather than the stale replay the drop-while-muted rule exists to avoid.
1127+
if (!outcome.startingUp) return@launch
1128+
1129+
val deadline = SystemClock.elapsedRealtime() + WAKE_WATCH_MS
1130+
while (SystemClock.elapsedRealtime() < deadline) {
1131+
delay(WAKE_POLL_MS)
1132+
// Both matter: the call may have ended, and the user may have switched away from the
1133+
// machine we are watching, which makes its readiness someone else's news.
1134+
if (!callActive || machineId != currentMachineId) return@launch
1135+
val awake =
1136+
try {
1137+
ConciergeClient.listMachines(p.baseUrl, SaiAuth.idToken() ?: p.token)
1138+
.firstOrNull { it.machineId == machineId }
1139+
?.isActive == true
1140+
} catch (e: Exception) {
1141+
log("wake: status poll failed — ${e.message}")
1142+
false // keep waiting; a blip is not a failed wake
1143+
}
1144+
if (awake) {
1145+
log("wake: machine is active")
1146+
say(WakePolicy.onWatchEnded(active = true, muted = saiMuted, audible = live != null))
1147+
return@launch
1148+
}
1149+
}
1150+
log("wake: gave up after ${WAKE_WATCH_MS / 1000}s — never reached active")
1151+
say(WakePolicy.onWatchEnded(active = false, muted = saiMuted, audible = live != null))
1152+
}
1153+
}
1154+
1155+
/**
1156+
* Act on a [WakePolicy] decision: speak the line verbatim, or log why not.
1157+
*
1158+
* The shared `speak:machine-state` kind is what makes a later line REPLACE an earlier one still held
1159+
* for the end of a turn; see rule 3 on [wakeMachine]. Same wrapper as `LiveVoiceChannel.say`, because
1160+
* these are `say` constants — and a paraphrase of "about a minute" is how a waking VM once sounded
1161+
* like a running task.
1162+
*/
1163+
private fun say(decision: WakeAnnouncement) {
1164+
when (decision) {
1165+
is WakeAnnouncement.Silent -> log("wake: not spoken — ${decision.why}")
1166+
is WakeAnnouncement.Speak ->
1167+
live?.injectNudge(
1168+
"speak:machine-state",
1169+
"[system] Say to the user, briefly and verbatim: \"${decision.line}\"")
10321170
}
10331171
}
10341172

@@ -1399,6 +1537,16 @@ class CallService : Service() {
13991537
* as "they're still talking" would make ending a call by voice impossible.
14001538
*/
14011539
private const val HANGUP_STRAGGLER_GUARD_MS = 600L
1540+
/**
1541+
* How long to keep watching for a woken machine to come up.
1542+
*
1543+
* `MACHINE_WAKING` promises "about a minute", so this is generous against that rather than tight:
1544+
* the failure it reports is real and unrecoverable, and calling one early on a machine that was
1545+
* merely slow would be the worse error.
1546+
*/
1547+
private const val WAKE_WATCH_MS = 3 * 60_000L
1548+
/** Poll interval while waiting for a wake. Cheap (`GET /machines`), and nothing pushes this. */
1549+
private const val WAKE_POLL_MS = 10_000L
14021550
private const val NOTIF_ID = 42
14031551
private const val REASON_NOTIF_ID = 43
14041552
/** Default "ask before delivering an update" threshold; overridable via StartParams (app setting). */

meta-android-app/app/src/main/java/com/meta/wearable/dat/externalsampleapps/cameraaccess/saispike/ConciergeClient.kt

Lines changed: 105 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,12 +54,61 @@ data class SessionBootstrap(
5454
)
5555

5656
/** A Sai machine (VM) the user can target, from GET /v1/agents/machines. */
57-
data class Machine(val machineId: String, val name: String?) {
57+
data class Machine(
58+
val machineId: String,
59+
val name: String?,
60+
/**
61+
* VM state as the server stores it: `active` · `hibernated` · `hibernating` · `wakingup`, or null
62+
* when the machine has never reported one — read that as offline.
63+
*
64+
* Only [isActive] is read at present, by the wake watcher. Deliberately kept as the server's own
65+
* string rather than an enum: a second spelling of these states is how the two drift apart, and
66+
* an unknown value must degrade to "not active" rather than fail to parse.
67+
*/
68+
val status: String? = null,
69+
/**
70+
* Whether a hibernated machine can be woken remotely at all — a property of where it is hosted,
71+
* not of what it is doing now.
72+
*
73+
* The reason this is not inferable from [status]: a `hibernated` machine with `canWake = false` is
74+
* asleep and staying that way, so announcing a wake for it promises a minute that never ends.
75+
*/
76+
val canWake: Boolean = false,
77+
) {
5878
/** Display label for the picker. */
5979
val label: String
6080
get() = name?.takeIf { it.isNotBlank() } ?: machineId
81+
82+
/** Up and usable. Anything else — including an unrecognised state — is not. */
83+
val isActive: Boolean
84+
get() = status == "active"
6185
}
6286

87+
/**
88+
* What `POST /v1/agents/wake` answered.
89+
*
90+
* Four fields because the caller has four different sentences to choose between, and only one of them
91+
* is true at a time. See [ConciergeClient.wakeMachine].
92+
*/
93+
data class WakeOutcome(
94+
/**
95+
* THIS call dispatched a wake. False for a machine that was already awake, already on its way, or
96+
* cannot be woken — and false is not the same as "nothing to say" in the second case.
97+
*/
98+
val waking: Boolean,
99+
/**
100+
* The machine is not usable yet but is coming up, whether or not we are the reason.
101+
*
102+
* **Branch on this, not [waking].** A machine already mid-wake answers `waking = false` — correctly,
103+
* since nothing was dispatched — and the user is still owed the "about a minute" line. It is also
104+
* false for a hibernated machine that cannot be woken, which is exactly when nothing should be said.
105+
*/
106+
val startingUp: Boolean,
107+
/** As stored, read server-side BEFORE the dispatch. Null when the field is absent. */
108+
val status: String?,
109+
val canWake: Boolean,
110+
)
111+
63112
object ConciergeClient {
64113
/**
65114
* List the user's Sai machines (like `sai machine` in the CLI) so the app can offer a picker.
@@ -84,7 +133,14 @@ object ConciergeClient {
84133
val arr = JSONObject(body).optJSONArray("machines") ?: return@withContext emptyList()
85134
(0 until arr.length()).map {
86135
val m = arr.getJSONObject(it)
87-
Machine(m.getString("machineId"), m.optString("name").ifEmpty { null })
136+
Machine(
137+
machineId = m.getString("machineId"),
138+
name = m.optString("name").ifEmpty { null },
139+
// Absent on a server older than 2026-08-20, which reads as "offline / cannot wake" —
140+
// and the wake path degrades to doing nothing rather than to guessing.
141+
status = m.optString("status").ifEmpty { null },
142+
canWake = m.optBoolean("canWake", false),
143+
)
88144
}
89145
}
90146

@@ -171,4 +227,51 @@ object ConciergeClient {
171227
}
172228
JSONObject(body)
173229
}
230+
231+
/**
232+
* Wake a hibernated machine, without sending it any work — `POST /v1/agents/wake`.
233+
*
234+
* The point of it having no payload: every other wake in the system rides a delivery, and a message
235+
* arriving during a running turn is folded INTO that turn. So waking by sending a throwaway "hello"
236+
* means the dummy and the user's next real request share one turn, and the dummy's completion ends
237+
* it out from under the real work. There is nothing to deliver at call bind anyway.
238+
*
239+
* Safe to call redundantly — the server no-ops on a machine that is already awake. Throws
240+
* [ConciergeHttpException] on a non-2xx; a 404 means the machine is not this account's.
241+
*/
242+
suspend fun wakeMachine(
243+
baseUrl: String,
244+
bearerToken: String,
245+
machineId: String,
246+
): WakeOutcome =
247+
withContext(Dispatchers.IO) {
248+
val conn =
249+
(URL("$baseUrl/v1/agents/wake").openConnection() as HttpURLConnection).apply {
250+
requestMethod = "POST"
251+
doOutput = true
252+
connectTimeout = 10_000
253+
readTimeout = 15_000
254+
applyCloudApiHeaders(bearerToken)
255+
setRequestProperty("Content-Type", "application/json")
256+
}
257+
conn.outputStream.use {
258+
it.write(JSONObject().put("machineId", machineId).toString().toByteArray())
259+
}
260+
val code = conn.responseCode
261+
val stream = if (code in 200..299) conn.inputStream else conn.errorStream
262+
val body = stream?.bufferedReader()?.use { it.readText() } ?: ""
263+
conn.disconnect()
264+
if (code !in 200..299) {
265+
throw ConciergeHttpException(code, "POST /v1/agents/wake failed: HTTP $code${body.take(300)}")
266+
}
267+
val o = JSONObject(body)
268+
WakeOutcome(
269+
waking = o.optBoolean("waking", false),
270+
// Absent on an older server: nothing is starting up as far as this client can tell, so the
271+
// wake path stays silent rather than announcing a minute it cannot vouch for.
272+
startingUp = o.optBoolean("startingUp", false),
273+
status = o.optString("status").ifEmpty { null },
274+
canWake = o.optBoolean("canWake", false),
275+
)
276+
}
174277
}

0 commit comments

Comments
 (0)