QVAC-23489 feat[api]: add ABot-World interactive world sessions to the SDK

[donriddo]

🎯 What problem does this PR solve?

• ABot-World shipped in @qvac/diffusion-cpp 0.19.0 (addon QVAC-21981 feat[api]: ABot-World interactive world sessions (walk + native scene creation) #3352), but nothing in @qvac/sdk exposed it — the capability was reachable only by require('@qvac/diffusion-cpp/world') from a bare/Node app.
• A walk session does not fit the plugin model contract as-is: it is stateful across calls, its load() reads a scene pack that may not exist yet, and its native teardown is synchronous down to a thread join.

📝 How does it solve it?

• modelConfig.mode: 'world' on the existing sdcpp-generation plugin (same addon package, following the video / upscale mode precedent), with two streaming ops: worldCreateScene builds a world from a prompt + first frame, worldStep walks it one generated block at a time and streams frames as they decode.
• Deferred activation. With sceneSrc the session activates at loadModel so a bad pack fails fast; without it, activation waits for the first worldStep after a world exists — because the caller is about to build one.
• Teardown waits for in-flight native work. The addon's unload() is synchronous down to processingThread_.join(), so entering it mid-job blocks the worker's event loop — every model and every RPC on it — until that job finishes. Measured: a mid-block unloadModel took 779 ms with a worst event-loop gap of 76 ms; entering the join would have stalled it for the full 779 ms.
• Scene packs are server-managed and ephemeral. Stored at a path derived from a hash of the model id (a caller-supplied id cannot steer the write) and deleted on unload, failed load and worker shutdown. Callers persist a world by keeping the bytes worldCreateScene returns and passing them back as sceneSrc. Replacement is staged and promoted with an atomic rename, so a failed generation leaves the previous world intact rather than leaving the model with none.
• Cancellation follows the engine's real semantics. A walk cancel is block-granular: the current block finishes internally, delivery stops, and the step rejects rather than resolving with a silently truncated block — the DiT has already committed that block to session history. Scene creation takes no abort predicate at all, so its concurrency slot is held until the native encode settles even if the caller disconnects.
• One job per model, rejected rather than queued: a walk is driven by live key input, so a backlog of stale keypresses is worse for the caller than a refusal it can drop.
• Delegated mode: 'world' is refused at loadModel rather than succeeding and failing on the first step, since the world ops have no delegated route.

🧪 How was it tested?

Static — typecheck, lint --max-warnings=0, format, contract:export + contract:check, and tsc over packages/sdk/e2e.

Suites — unit 114/114 files; bare 131/131 tests / 523/523 asserts; test:security and test:security:bare.

New coverage

• test/bare/sdcpp-world-ops.test.ts — drives the ops through an injected native session: run() follows the response to terminal state (not just dispatch), teardown is not entered while a job is tracked, abort never emits done: true, and the model slot is held until settle even when the consumer abandons the stream.
• test/bare/sdcpp-world-plugin.test.ts — companion resolution, world-only field rejection, eager vs deferred activation, staged replacement and rollback, ephemeral cleanup, and a traversal-shaped model id not steering the scene path.
• test/unit/world-{schemas,client,concurrency}.test.ts — config accept/reject matrix, key normalisation across array/object/mask forms, both client result factories including error paths, and the installed reject-on-overlap admission policy.

Hardware (RTX 5080, 448x256 low-VRAM tier) — loadModel → worldCreateScene → worldStep xN → unloadModel:

check	result
scene pack	9,458,120 bytes, 2,811 ms
first / second block frames	9 / 12
actionMask for W+L	129
frame dimensions	448x256
concurrent step	rejected, structured error
cancel mid-block	rejected Job cancelled; session reloaded and walked again
unloadModel mid-block	779 ms, worst event-loop gap 76 ms
VRAM after unload	back to 18 MiB; managed pack deleted

SDK E2E (desktop consumer, RTX 5080, real models via the registry) — 8/8 passing. npx qvac-test run:local:desktop --filter world, covering scene creation, first/second block frame counts, step-before-scene, invalid key and dimensions, overlap rejection and cancel-then-reload. Peak RSS across the batch 3,791 MB.

Running them found a defect in the tests themselves: world-step-before-scene-fails passed only when it happened to run before any world was created, because the resource is shared through ensureLoaded. It evicts first now, so the precondition is real rather than dependent on test ordering (7/8 → 8/8).

🔌 API Changes

const modelId = await loadModel({
  modelSrc: ABOT_WORLD_0_5B_Q8_0,
  modelType: 'sdcpp-generation',
  modelConfig: {
    mode: 'world',
    taehvModelSrc: ABOT_WORLD_0_5B_LF_VAE,
    t5XxlModelSrc: UMT5_XXL_ENC_Q8_0,
    vaeModelSrc: ABOT_WORLD_0_5B_LF_VAE_F16,
    world: { kvCache: true, frameJpegQuality: 85 }
  }
})

// Once per world: prompt + first frame -> scene pack, returned for reuse via sceneSrc.
const { scene } = worldCreateScene({ modelId, prompt, image })
await scene

// Walk: one generated block per call, frames stream as they decode.
const { frameStream } = worldStep({ modelId, keys: ['W', 'L'] })
for await (const frame of frameStream) render(frame)

keys also accepts a key-state object ({ W: true }) or a raw 8-bit mask, so a keyboard handler can pass its state straight through.

📋 Open items

• Darwin desktop E2E registration. Mac CI has OOM-killed on large diffusion model sets in this area (mac-mini-m4-gpu, addon lane), and the ABot set is 13.3 GB. A Darwin skip looks warranted, but the desktop consumer currently has no platform-conditional logic at all, so this would be the first — flagging rather than introducing that pattern unilaterally.
• Hardware validation covered the 448x256 correctness tier; the 832x480 full-fidelity/performance run on the 2x RTX 5090 host is still outstanding and non-blocking.
• Delegated inference for the diffusion-family ops is out of scope here and wants its own ticket: only completionStream has a delegated route today, so the mobile delegation guidance in sdcppConfigSchema cannot work as written.

---

• To see the specific tasks where the Asana app for GitHub is being used, see below:
  • https://app.asana.com/0/0/1217366184529552

[github-actions]

License compliance — findings detected (warn-only)

Critical: 0 · High: 1 · Medium: 0

Dependency	License	Scope	Severity	Outcome
@qvac/diffusion-cpp@^0.19.0	(none detected)	runtime	High	blocks

How to resolve a blocking finding:

• Remove or replace the disallowed dependency, or
• If the license is genuinely acceptable, run the compliance SKILL and record the decision in .github/license-allowlist.yml (CODEOWNERS-reviewed), or
• For a one-off, a maintainer can apply the license-override label (High findings only; Critical cannot be overridden).

Warn-only (shadow) mode — this check does not block merges yet.

Updated automatically by the canonical license compliance workflow.

NOTICE presence (advisory)

Missing NOTICE (advisory, does not block):

• ./.github/actions/release-merge-guard
• ./docs/website
• ./packages/ggml-coload-smoke
• ./packages/fabric/test/integration
• ./packages/inference-addon-cpp/mobile
• ./packages/sdk/e2e
• ./packages/llm-llamacpp/benchmarks/performance
• ./packages/llm-llamacpp/benchmarks/server
• ./packages/vla-ggml/sim/server
• ./packages/embed-llamacpp/benchmarks/performance
• ./packages/embed-llamacpp/benchmarks/server
• ./packages/asr-ggml/benchmarks/server

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​qvac/​diffusion-cpp@​0.19.0

View full report

[github-actions]

Review Status

Current Status: ❌ PENDING
Approvals so far: none

Pending reviews: Needs 1 Management or Team Lead, and 1 more from Management, Team Lead, or Member.