Stage 5 generate, user status, and housekeeping migration

[Leechael]

Summary

• add StreamGenerate protocol codec, transport facade, and split generate client code
• route GetUserStatus and quota calls through CallRPC
• add housekeeping RPC encoders and debug housekeeping command

Verification

• make ci
• make build
• prek

Notes

• upload.go still uses direct HTTP calls; Stage 5 plan marks upload out of scope despite the facade-count checklist mentioning zero bypasses.

---

Summary by cubic

Adds StreamGenerate protocol codec and streaming transport, migrates the generate flow to a clean protocol/transport split, and fixes incremental parsing of streaming frames. Routes user status and quota through the unified RPC path and extends housekeeping RPCs with a debug housekeeping command.

• New Features

  • StreamGenerate codec and media extractors in internal/client/protocol/rpcs with fixtures/tests; extractors support new and old video/audio paths.
  • Streaming transport in internal/client/transport with CallStreamGenerate; maps 429 to RateLimitError.
  • Housekeeping RPC encoders (heartbeat, UI heartbeat, prefs, bulk log, list gems, log event/model selection) and cmd/debug housekeeping.
  • GetUserStatus decoder with tests; protocol fixtures added.

• Refactors

  • Replaced monolithic internal/client/generate.go with generate_*; moved wire encode/decode to protocol/rpcs.
  • Isolated request building and streaming parser; parser now handles chunked frames incrementally; centralized mode resolution.
  • Routed FetchUserStatus and quota calls through CallRPC/batch, removing manual HTTP.
  • Left upload.go on direct HTTP (Stage 5 out of scope).

^{Written for commit f5c4b06. Summary will update on new commits. Review in cubic}

[cubic-dev-ai]

3 issues found across 38 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="internal/client/protocol/testdata/stream_generate_basic_response.txt">

<violation number="1" location="internal/client/protocol/testdata/stream_generate_basic_response.txt:2">
P2: Fix the length prefix to match the frame size; the current `310` prefix makes this response look incomplete to the stream parser.</violation>
</file>

<sub>Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Fix the length prefix to match the frame size; the current 310 prefix makes this response look incomplete to the stream parser.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At internal/client/protocol/testdata/stream_generate_basic_response.txt, line 2:

<comment>Fix the length prefix to match the frame size; the current `310` prefix makes this response look incomplete to the stream parser.</comment>

<file context>
@@ -0,0 +1,3 @@
+)]}'
+310
+[["wrb.fr",null,"[null,[\"c_000000000000001\",\"r_000000000000001\",\"rcid_000000000000001\"],null,null,[[\"rcid_000000000000001\",[\"Sample assistant response.\"]]],null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,null,\"completion_context_000000000000001\"]"]]
</file context>

[Leechael]

Followed up on cubic review:

• fixed stream parsing to consume frames incrementally instead of reparsing the accumulated response
• fixed non-EOF read errors so partial output no longer hides stream failures
• added regression tests for chunked parsing, read-error handling, and the checked-in StreamGenerate fixture

I did not change the stream_generate_basic_response.txt length prefix: in this parser's framing convention the declared length includes the newline after the length line and the trailing frame newline. TestSampleFixtureLengthPrefixes and the new TestParseStreamResponse_ProtocolFixture both cover this fixture.

Verification:

• make ci
• prek

[Leechael]

Followed up on the latest review feedback.

P1 fixes:

• made isQueueingFrame a false stub until a real queueing HAR confirms the exact path
• made the test-only parseEnvelope wrapper panic on decoder errors instead of swallowing them
• removed the unused TemporaryChat encoder branch
• removed the dead hasCid parameter from buildInnerRequest
• removed dead parseUserStatus
• added MyActivity iframe token-risk notes for both dI8W6e and TFNzk
• added protocol header blocks to all housekeeping RPC files
• rewrote the initial fixture commit so get_user_status_basic.txt is framed from the first commit

P2 fixes included:

• added old-path music extraction coverage
• documented why StreamGenerate uses URL-path instead of Source-path
• cleaned up DecodeGetUserStatus trim handling
• added account status semantics comment
• asserted empty TierFlags in the fixture test
• added multi-counter bulk log coverage

Verification:

• go test ./...
• checked the first fixture commit with go test ./internal/client/protocol
• make ci
• prek