feat: expose Web SDK streaming APIs and describeImage in wrapper libraries#20
Merged
feat: expose Web SDK streaming APIs and describeImage in wrapper libraries#20
Conversation
- Add libraries/API_PARITY.md documenting cross-platform availability:
- Core features, streaming variants, image APIs, Chrome-only APIs (detectLanguage,
write, writeStreaming), model management, and Android-only APIs in a single table
- expo-ondevice-ai: add TextStreamChunk, SummarizeStreamOptions, TranslateStreamOptions,
RewriteStreamOptions, DescribeImageOptions/Result types and corresponding functions
(summarizeStreaming, translateStreaming, rewriteStreaming, describeImage) using the
existing EventSubscription pattern from chatStream
- react-native-ondevice-ai: add NitroTextStreamChunk + NitroDescribeImageOptions/Result
to the Nitro spec; add listener pairs for summarize/translate/rewrite streaming;
add describeImage method; mirror new public types and functions in types.ts and index.ts
- detectLanguage / write / writeStreaming documented as Chrome-only in API_PARITY.md
Closes #12
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (206)
📒 Files selected for processing (5)
📝 WalkthroughWalkthroughThis PR adds token-by-token streaming APIs (summarizeStreaming, translateStreaming, rewriteStreaming) with Changes
Sequence DiagramsequenceDiagram
participant Client as JavaScript Client
participant Module as Wrapper Module (Index.ts)
participant Listener as Event Listener
participant Native as Native Implementation
Client->>Module: summarizeStreaming(text, {onChunk})
Module->>Module: registerListener(onSummarizeStreamChunk)
Module->>Native: ExpoOndeviceAiModule.summarizeStreaming(text)
Native->>Native: Process streaming text chunks
Native->>Listener: emit onSummarizeStreamChunk({delta, accumulated, isFinal})
Listener->>Module: listener callback received
Module->>Module: convert to TextStreamChunk
Module->>Client: invoke onChunk(chunk)
Native->>Module: return Promise<SummarizeResult>
Module->>Module: cleanup listener
Module->>Module: zero-timeout flush
Module->>Client: return final result
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~30 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request successfully exposes new streaming APIs (summarizeStreaming, translateStreaming, rewriteStreaming) and the describeImage API in the expo-ondevice-ai and react-native-ondevice-ai wrapper libraries. The addition of API_PARITY.md is a great improvement for documenting cross-platform feature availability. The code changes are well-structured and follow existing patterns. I've identified a few opportunities to improve maintainability by reducing code duplication in the new streaming function implementations and addressing some type casting that could be made clearer. Overall, this is a solid contribution that expands the library's capabilities.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@libraries/API_PARITY.md`:
- Around line 51-52: The note incorrectly suggests using device 'locale' as a
replacement for the API's language-detection function; update the text to
clearly state that device locale (e.g., "locale") reflects user/device
preferences and is not equivalent to the API's content-based language detection
(e.g., "detectLanguage(text)"); explicitly recommend third-party libraries (like
react-native-mlkit) or sending text to the server-side detectLanguage(text)
endpoint for accurate detection of arbitrary input, and remove or reword any
phrasing that implies locale is a substitute for detectLanguage(text).
- Around line 27-29: The API parity matrix shows streaming methods
(summarizeStreaming, translateStreaming, rewriteStreaming with onChunk
callbacks) present in expo-ondevice-ai and react-native-ondevice-ai but missing
from flutter_ondevice_ai; either implement matching public methods in the
Flutter wrapper (add public methods named summarizeStreaming,
translateStreaming, rewriteStreaming that accept the same options and onChunk
callback semantics as the other wrappers) or delay the rollout so the new APIs
are not listed until all three wrappers are identical—update the Flutter wrapper
code to expose those exact method names and signatures or remove/gate the
entries from the matrix to keep wrappers in lockstep.
In `@libraries/expo-ondevice-ai/src/index.ts`:
- Around line 196-206: The stream listeners (see summarizeStreaming,
translateStreaming, rewriteStreaming) are global per-event and will deliver
every onSummarizeStreamChunk event to all subscribers; fix by making streams
request-scoped: when starting a stream from those functions, generate/provide a
unique streamId and ensure the native emitter emits TextStreamChunk objects with
that streamId, then in the addListener callback only invoke options.onChunk when
chunk.streamId === localStreamId (or alternatively, if you cannot change native
emissions, enforce single-concurrent-stream by refusing to start a new
summarizeStreaming/translateStreaming/rewriteStreaming while a subscription
exists). Also ensure subscriptions are removed on completion/error and propagate
the streamId check to the other event names used in the file.
- Around line 197-205: The JS wrapper is calling and subscribing to native
symbols that don't exist; either add the missing native APIs to the iOS and
Android modules or stop using them in the wrapper. Implement native methods
named summarizeStreaming, translateStreaming, rewriteStreaming, and
describeImage on the native module (ExpoOndeviceAiModule), and emit events named
onSummarizeStreamChunk, onTranslateStreamChunk, and onRewriteStreamChunk (in
addition to the existing onChatStreamChunk and onModelDownloadProgress), or
alternatively remove/replace calls to
summarizeStreaming/translateStreaming/rewriteStreaming/describeImage and
subscriptions to
onSummarizeStreamChunk/onTranslateStreamChunk/onRewriteStreamChunk in index.ts
so the wrapper only uses native methods/events that actually exist.
In `@libraries/react-native-ondevice-ai/src/index.ts`:
- Around line 431-455: The translateStreaming path uses an empty string for
omitted sourceLanguage, causing divergence from translate(); update
translateStreaming to mirror translate() by mapping options.sourceLanguage ??
'en' (or use the same option-mapping helper if one exists) before calling
AI.instance.translateStreaming so both streaming and non-streaming translate
calls use the same default sourceLanguage convention; adjust the creation of the
options passed to AI.instance.translateStreaming in translateStreaming to use
that mapped value and ensure the onChunk listener logic remains unchanged.
In `@libraries/react-native-ondevice-ai/src/specs/OndeviceAi.nitro.ts`:
- Around line 209-213: The current streaming contract (NitroTextStreamChunk)
lacks a stream identifier so concurrent streams cannot be demultiplexed; update
the contract and listener API to include a unique streamId on
NitroTextStreamChunk (and any equivalent chunk interfaces referenced around the
301-332 region), and propagate that id through the global add/remove listener
pair (or alternatively change the API to enforce a single active stream per
feature by preventing addListener if a stream is already active). Specifically
modify the NitroTextStreamChunk interface to include streamId and update the
corresponding addListener/removeListener handlers and any emitters/dispatchers
to attach and check streamId so JS consumers can route chunks to the correct
in-flight stream.
- Around line 301-338: The native bridge is missing implementations for the ten
newly-declared methods, causing runtime failures; implement summarizeStreaming,
addSummarizeStreamListener, removeSummarizeStreamListener, translateStreaming,
addTranslateStreamListener, removeTranslateStreamListener, rewriteStreaming,
addRewriteStreamListener, removeRewriteStreamListener, and describeImage in both
HybridOndeviceAi.swift and HybridOndeviceAi.kt. For each streaming method
(summarizeStreaming, translateStreaming, rewriteStreaming) implement a native
function that starts the operation on the existing on-device engine, returns a
Promise/Future that resolves with
NitroSummarizeResult/NitroTranslateResult/NitroRewriteResult when the stream
completes, and emits intermediate NitroTextStreamChunk objects via the existing
event/listener system; for each add*StreamListener and remove*StreamListener
implement listener registration/unregistration that forwards native chunk events
to JS (use the same event names and callback wiring used elsewhere in this
bridge). For describeImage implement a synchronous/async call that calls the
image description routine on the native engine and resolves with
NitroDescribeImageResult. Ensure method signatures exactly match the Nitro spec
names so JS calls bind correctly and reuse existing error handling/logging
patterns in the bridge.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 69f81f86-14af-44dd-8ac0-9746cbf84abc
📒 Files selected for processing (6)
libraries/API_PARITY.mdlibraries/expo-ondevice-ai/src/index.tslibraries/expo-ondevice-ai/src/types.tslibraries/react-native-ondevice-ai/src/index.tslibraries/react-native-ondevice-ai/src/specs/OndeviceAi.nitro.tslibraries/react-native-ondevice-ai/src/types.ts
Comment on lines
+27
to
+29
| | `summarizeStreaming(text, options?)` | ✅ | ✅ | 🚧 | `onChunk` callback | | ||
| | `translateStreaming(text, options)` | ✅ | ✅ | 🚧 | `onChunk` callback | | ||
| | `rewriteStreaming(text, options)` | ✅ | ✅ | 🚧 | `onChunk` callback | |
There was a problem hiding this comment.
This matrix still documents wrapper API drift.
If this table is accurate, flutter_ondevice_ai is still missing the new public APIs while Expo and React Native ship them. That conflicts with the repo invariant of keeping wrapper APIs identical, so these additions should land in Flutter in the same release or the rollout should stay gated until the wrappers are back in lockstep.
Based on learnings, "All three libraries (expo-ondevice-ai, react-native-ondevice-ai, flutter_ondevice_ai) MUST expose identical public APIs."
Also applies to: 36-37
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libraries/API_PARITY.md` around lines 27 - 29, The API parity matrix shows
streaming methods (summarizeStreaming, translateStreaming, rewriteStreaming with
onChunk callbacks) present in expo-ondevice-ai and react-native-ondevice-ai but
missing from flutter_ondevice_ai; either implement matching public methods in
the Flutter wrapper (add public methods named summarizeStreaming,
translateStreaming, rewriteStreaming that accept the same options and onChunk
callback semantics as the other wrappers) or delay the rollout so the new APIs
are not listed until all three wrappers are identical—update the Flutter wrapper
code to expose those exact method names and signatures or remove/gate the
entries from the matrix to keep wrappers in lockstep.
Member
Author
There was a problem hiding this comment.
Flutter is intentionally behind in this PR — streaming and describeImage will be added in a follow-up PR. The API_PARITY.md table accurately reflects the current state.
There was a problem hiding this comment.
@hyochan, understood — the 🚧 markers in the table clearly convey the planned-but-not-yet-landed status for Flutter, so the current state is accurately documented. I'll update my understanding of the API parity policy accordingly.
(ᓀ ᓀ) ~♪
✏️ Learnings added
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 20
File: libraries/API_PARITY.md:27-29
Timestamp: 2026-03-21T09:12:17.035Z
Learning: In the `hyodotdev/locanara` repo, temporary API divergence between wrapper libraries (expo-ondevice-ai, react-native-ondevice-ai, flutter_ondevice_ai) is acceptable when the gap is intentional and documented with 🚧 markers in `libraries/API_PARITY.md`, indicating a follow-up PR is planned. The invariant that all three wrappers expose identical APIs is the long-term goal, not a strict per-PR requirement.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
⛔ Learnings removed
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: All three libraries (expo-ondevice-ai, react-native-ondevice-ai, flutter_ondevice_ai) MUST expose identical public APIs
🧠 Learnings used
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: All three libraries (expo-ondevice-ai, react-native-ondevice-ai, flutter_ondevice_ai) MUST expose identical public APIs
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:61-79
Timestamp: 2026-02-22T08:44:32.446Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the `checkAvailability()` function intentionally uses a lenient fallback in its catch block (`return hasAPI(api)`) because Chrome Built-in AI APIs are evolving and `availability()` can timeout on first load. This prevents features from being incorrectly locked, while actual API failures are still caught and handled by each feature method.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:28-34
Timestamp: 2026-02-23T14:50:03.100Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the cached Chrome Built-in AI instances (cachedSummarizer, cachedLanguageModel, cachedRewriter, cachedWriter, cachedTranslators) intentionally use `any` type instead of `types/dom-chromium-ai` because Chrome Built-in AI APIs are experimental and changing rapidly. Using `any` with runtime checks is more practical than depending on external type packages that may lag behind API changes.
<!-- [/add_learning]
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/src/Locanara.ts:576-592
Timestamp: 2026-02-22T10:24:08.605Z
Learning: In `packages/web/src/Locanara.ts`, the standalone web SDK targets Chrome 138+ which uses delta mode for `promptStreaming`, so cumulative-to-delta normalization is not required. The expo module (`libraries/expo-ondevice-ai`) has auto-detection as a safety net for broader version support.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-23T14:32:41.094Z
Learning: In `packages/web/src/Locanara.ts`, the SDK's `chatStreaming` method internally handles both cumulative and delta modes from Chrome's `promptStreaming` API, always yielding delta chunks to consumers. This normalization happens at the SDK level, so example apps and other consumers can safely use simple concatenation (`response += chunk`) without worrying about the streaming mode.
Comment on lines
+51
to
+52
| If you need language detection on mobile, consider using a third-party library like | ||
| `react-native-mlkit` or the device's built-in locale detection. |
There was a problem hiding this comment.
locale detection is not a replacement for detectLanguage(text).
The device locale only reflects user/device preference, not the language of arbitrary input text. This note points consumers at a different capability than the API in this table.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libraries/API_PARITY.md` around lines 51 - 52, The note incorrectly suggests
using device 'locale' as a replacement for the API's language-detection
function; update the text to clearly state that device locale (e.g., "locale")
reflects user/device preferences and is not equivalent to the API's
content-based language detection (e.g., "detectLanguage(text)"); explicitly
recommend third-party libraries (like react-native-mlkit) or sending text to the
server-side detectLanguage(text) endpoint for accurate detection of arbitrary
input, and remove or reword any phrasing that implies locale is a substitute for
detectLanguage(text).
Member
Author
There was a problem hiding this comment.
Correct — locale is not a replacement for detectLanguage. The API_PARITY.md notes this as a known gap. Language detection from text is tracked separately.
There was a problem hiding this comment.
@hyochan, understood — thanks for the clarification. Since detectLanguage from text content is a known gap on mobile platforms, would you like me to open a follow-up GitHub issue to formally track adding mobile-native language detection support (or a cross-platform alternative) to the wrapper libraries? That way it stays visible alongside #12.
(ʘ‿ʘ✿)
✏️ Learnings added
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 20
File: libraries/API_PARITY.md:51-52
Timestamp: 2026-03-21T09:12:15.737Z
Learning: In `libraries/API_PARITY.md`, the absence of `detectLanguage(text)` in Expo/React Native/Flutter wrappers is a known, intentional gap. Device locale is explicitly NOT a substitute. Mobile language detection from arbitrary text content is tracked separately as a future work item.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: All three libraries (expo-ondevice-ai, react-native-ondevice-ai, flutter_ondevice_ai) MUST expose identical public APIs
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:61-79
Timestamp: 2026-02-22T08:44:32.446Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the `checkAvailability()` function intentionally uses a lenient fallback in its catch block (`return hasAPI(api)`) because Chrome Built-in AI APIs are evolving and `availability()` can timeout on first load. This prevents features from being incorrectly locked, while actual API failures are still caught and handled by each feature method.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:28-34
Timestamp: 2026-02-23T14:50:03.100Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the cached Chrome Built-in AI instances (cachedSummarizer, cachedLanguageModel, cachedRewriter, cachedWriter, cachedTranslators) intentionally use `any` type instead of `types/dom-chromium-ai` because Chrome Built-in AI APIs are experimental and changing rapidly. Using `any` with runtime checks is more practical than depending on external type packages that may lag behind API changes.
<!-- [/add_learning]
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/src/Locanara.ts:576-592
Timestamp: 2026-02-22T10:24:08.605Z
Learning: In `packages/web/src/Locanara.ts`, the standalone web SDK targets Chrome 138+ which uses delta mode for `promptStreaming`, so cumulative-to-delta normalization is not required. The expo module (`libraries/expo-ondevice-ai`) has auto-detection as a safety net for broader version support.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/README.md:41-41
Timestamp: 2026-02-23T14:32:47.535Z
Learning: The Chrome Built-in AI `LanguageModel.availability()` API can return both `'readily'` and `'available'` depending on the Chrome version. Earlier versions used `'readily'`, while current versions (Chrome 138+) use `'available'`. The README comment in `packages/web/README.md` reflects the behavior in newer Chrome versions.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-23T14:32:41.094Z
Learning: In `packages/web/src/Locanara.ts`, the SDK's `chatStreaming` method internally handles both cumulative and delta modes from Chrome's `promptStreaming` API, always yielding delta chunks to consumers. This normalization happens at the SDK level, so example apps and other consumers can safely use simple concatenation (`response += chunk`) without worrying about the streaming mode.
Comment on lines
+196
to
+206
| if (options?.onChunk) { | ||
| subscription = ( | ||
| ExpoOndeviceAiModule as unknown as { | ||
| addListener: ( | ||
| name: string, | ||
| listener: (chunk: TextStreamChunk) => void, | ||
| ) => EventSubscription; | ||
| } | ||
| ).addListener('onSummarizeStreamChunk', (chunk: TextStreamChunk) => { | ||
| options.onChunk!(chunk); | ||
| }); |
There was a problem hiding this comment.
These stream listeners are not request-scoped.
If two summarizeStreaming() / translateStreaming() / rewriteStreaming() calls overlap, every active listener on that event name will receive every chunk. That will mix responses between callers. Please carry a stream/request ID in the emitted chunk or reject concurrent streams per feature.
Also applies to: 235-245, 274-284
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libraries/expo-ondevice-ai/src/index.ts` around lines 196 - 206, The stream
listeners (see summarizeStreaming, translateStreaming, rewriteStreaming) are
global per-event and will deliver every onSummarizeStreamChunk event to all
subscribers; fix by making streams request-scoped: when starting a stream from
those functions, generate/provide a unique streamId and ensure the native
emitter emits TextStreamChunk objects with that streamId, then in the
addListener callback only invoke options.onChunk when chunk.streamId ===
localStreamId (or alternatively, if you cannot change native emissions, enforce
single-concurrent-stream by refusing to start a new
summarizeStreaming/translateStreaming/rewriteStreaming while a subscription
exists). Also ensure subscriptions are removed on completion/error and propagate
the streamId check to the other event names used in the file.
Member
Author
There was a problem hiding this comment.
Valid concern for concurrent calls. In practice, these streaming APIs are called sequentially in the UI layer. The listener is scoped by the try/finally block and removed after each call completes.
There was a problem hiding this comment.
@hyochan, understood — the try/finally guarantees cleanup after each call (success or error), and sequential usage in the UI layer makes concurrent overlap unlikely. The concern is really about downstream consumers who call these APIs concurrently without knowing about the shared listener, since there's no runtime enforcement.
If you'd like a lightweight safety net without adding a streamId, a simple in-flight guard (one let activeSummarizeStream = false flag per feature) would throw or queue concurrent calls rather than silently mixing chunks. But if the sequential contract is documented or enforced at a higher layer, that's a reasonable call to defer.
✏️ Learnings added
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 20
File: libraries/expo-ondevice-ai/src/index.ts:196-206
Timestamp: 2026-03-21T09:12:22.328Z
Learning: In `libraries/expo-ondevice-ai/src/index.ts`, the streaming APIs (`summarizeStreaming`, `translateStreaming`, `rewriteStreaming`) use a shared global event listener per feature (e.g. `onSummarizeStreamChunk`) scoped only by the `try/finally` block that removes the subscription after each call. Concurrent calls are intentionally avoided at the UI layer rather than enforced at the API level. This matches the same pattern used by `chatStream` and `downloadModel` in the same file. There is no request-scoped `streamId` — the design relies on sequential invocation.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-23T14:32:41.094Z
Learning: In `packages/web/src/Locanara.ts`, the SDK's `chatStreaming` method internally handles both cumulative and delta modes from Chrome's `promptStreaming` API, always yielding delta chunks to consumers. This normalization happens at the SDK level, so example apps and other consumers can safely use simple concatenation (`response += chunk`) without worrying about the streaming mode.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:32-34
Timestamp: 2026-02-23T14:50:12.628Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the unbounded growth of `cachedTranslators` Map and lack of cleanup for `cachedWriter`/`cachedRewriter` is acceptable for now because users typically translate between only a few language pairs in practice, keeping the cache small. A cleanup mechanism with destroy() and LRU eviction may be added in a future PR if needed.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:28-34
Timestamp: 2026-02-23T14:50:03.100Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the cached Chrome Built-in AI instances (cachedSummarizer, cachedLanguageModel, cachedRewriter, cachedWriter, cachedTranslators) intentionally use `any` type instead of `types/dom-chromium-ai` because Chrome Built-in AI APIs are experimental and changing rapidly. Using `any` with runtime checks is more practical than depending on external type packages that may lag behind API changes.
<!-- [/add_learning]
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/src/Locanara.ts:576-592
Timestamp: 2026-02-22T10:24:08.605Z
Learning: In `packages/web/src/Locanara.ts`, the standalone web SDK targets Chrome 138+ which uses delta mode for `promptStreaming`, so cumulative-to-delta normalization is not required. The expo module (`libraries/expo-ondevice-ai`) has auto-detection as a safety net for broader version support.
Comment on lines
+197
to
+205
| subscription = ( | ||
| ExpoOndeviceAiModule as unknown as { | ||
| addListener: ( | ||
| name: string, | ||
| listener: (chunk: TextStreamChunk) => void, | ||
| ) => EventSubscription; | ||
| } | ||
| ).addListener('onSummarizeStreamChunk', (chunk: TextStreamChunk) => { | ||
| options.onChunk!(chunk); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
fd 'ExpoOndeviceAiModule\.(swift|kt)$' libraries/expo-ondevice-ai | while read -r file; do
echo "== $file =="
rg -n 'onSummarizeStreamChunk|onTranslateStreamChunk|onRewriteStreamChunk|summarizeStreaming|translateStreaming|rewriteStreaming|describeImage' "$file"
echo
doneRepository: hyodotdev/locanara
Length of output: 164
🏁 Script executed:
cat -n libraries/expo-ondevice-ai/android/src/main/java/expo/modules/ondeviceai/ExpoOndeviceAiModule.ktRepository: hyodotdev/locanara
Length of output: 16515
🏁 Script executed:
fd 'HybridOndeviceAi\.swift$' libraries/expo-ondevice-aiRepository: hyodotdev/locanara
Length of output: 44
🏁 Script executed:
fd 'OndeviceAi\.nitro\.ts$' libraries/expo-ondevice-aiRepository: hyodotdev/locanara
Length of output: 44
🏁 Script executed:
head -n 320 libraries/expo-ondevice-ai/src/index.ts | tail -n 125Repository: hyodotdev/locanara
Length of output: 3667
🏁 Script executed:
find libraries/expo-ondevice-ai -type f \( -name "*.swift" -o -name "*.ts" \) | grep -E "(spec|swift|Hybrid)"Repository: hyodotdev/locanara
Length of output: 297
🏁 Script executed:
rg -l 'summarizeStreaming|translateStreaming|rewriteStreaming|describeImage' libraries/expo-ondevice-aiRepository: hyodotdev/locanara
Length of output: 216
🏁 Script executed:
cat -n libraries/expo-ondevice-ai/ios/ExpoOndeviceAiModule.swiftRepository: hyodotdev/locanara
Length of output: 10440
🏁 Script executed:
find libraries/expo-ondevice-ai -name "*.nitro.ts" -o -name "*spec*.ts"Repository: hyodotdev/locanara
Length of output: 44
The wrapper calls native methods and listens to events that are not defined in either the iOS or Android module.
The following methods are missing from both ExpoOndeviceAiModule.swift (iOS) and ExpoOndeviceAiModule.kt (Android):
summarizeStreaming()translateStreaming()rewriteStreaming()describeImage()
The following events are missing from both modules:
onSummarizeStreamChunkonTranslateStreamChunkonRewriteStreamChunk
Both native modules currently define only onChatStreamChunk and onModelDownloadProgress (line 12 in Swift, line 53 in Kotlin). The wrapper will fail at runtime when attempting to call these undefined methods or subscribe to undefined events.
These methods and events must be added to both native implementations to match the wrapper's expectations, or the wrapper must be revised to use only the methods and events that actually exist in the native modules.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libraries/expo-ondevice-ai/src/index.ts` around lines 197 - 205, The JS
wrapper is calling and subscribing to native symbols that don't exist; either
add the missing native APIs to the iOS and Android modules or stop using them in
the wrapper. Implement native methods named summarizeStreaming,
translateStreaming, rewriteStreaming, and describeImage on the native module
(ExpoOndeviceAiModule), and emit events named onSummarizeStreamChunk,
onTranslateStreamChunk, and onRewriteStreamChunk (in addition to the existing
onChatStreamChunk and onModelDownloadProgress), or alternatively remove/replace
calls to summarizeStreaming/translateStreaming/rewriteStreaming/describeImage
and subscriptions to
onSummarizeStreamChunk/onTranslateStreamChunk/onRewriteStreamChunk in index.ts
so the wrapper only uses native methods/events that actually exist.
Member
Author
There was a problem hiding this comment.
The native implementations are in progress. This PR adds the JS/TS API surface first, with native module implementations following in subsequent PRs.
There was a problem hiding this comment.
@hyochan, understood — thanks for the context! Shipping the JS/TS API surface ahead of the native implementations is a reasonable incremental approach.
One suggestion worth considering: to prevent silent runtime failures if anyone calls these new APIs before the native side is ready, you could add a lightweight guard in each stub that throws a descriptive Error (e.g. "summarizeStreaming: native implementation not yet available"). This makes the "not yet implemented" state explicit rather than surfacing a cryptic native call failure. It also serves as a self-documenting hint for anyone exploring the API surface in the meantime.
🧠 Learnings used
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-23T14:32:41.094Z
Learning: In `packages/web/src/Locanara.ts`, the SDK's `chatStreaming` method internally handles both cumulative and delta modes from Chrome's `promptStreaming` API, always yielding delta chunks to consumers. This normalization happens at the SDK level, so example apps and other consumers can safely use simple concatenation (`response += chunk`) without worrying about the streaming mode.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-22T10:24:15.711Z
Learning: Chrome 138+ uses delta mode for promptStreaming API, where each chunk contains only the new text (delta) rather than the cumulative response. This means simple concatenation (`response += chunk`) is the correct approach when targeting Chrome 138+ as the minimum requirement.
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: Update both iOS (`ios/HybridOndeviceAi.swift`) and Android (`android/.../HybridOndeviceAi.kt`) native implementations when modifying Nitro Module API
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:32-34
Timestamp: 2026-02-23T14:50:12.628Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the unbounded growth of `cachedTranslators` Map and lack of cleanup for `cachedWriter`/`cachedRewriter` is acceptable for now because users typically translate between only a few language pairs in practice, keeping the cache small. A cleanup mechanism with destroy() and LRU eviction may be added in a future PR if needed.
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: Start Nitro Module API changes from the spec file (`src/specs/OndeviceAi.nitro.ts`) before modifying native implementations
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:28-34
Timestamp: 2026-02-23T14:50:03.100Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the cached Chrome Built-in AI instances (cachedSummarizer, cachedLanguageModel, cachedRewriter, cachedWriter, cachedTranslators) intentionally use `any` type instead of `types/dom-chromium-ai` because Chrome Built-in AI APIs are experimental and changing rapidly. Using `any` with runtime checks is more practical than depending on external type packages that may lag behind API changes.
<!-- [/add_learning]
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: All three libraries (expo-ondevice-ai, react-native-ondevice-ai, flutter_ondevice_ai) MUST expose identical public APIs
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:61-79
Timestamp: 2026-02-22T08:44:32.446Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the `checkAvailability()` function intentionally uses a lenient fallback in its catch block (`return hasAPI(api)`) because Chrome Built-in AI APIs are evolving and `availability()` can timeout on first load. This prevents features from being incorrectly locked, while actual API failures are still caught and handled by each feature method.
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: Keep API surface identical across all platforms (iOS, Android, Web)
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/src/Locanara.ts:576-592
Timestamp: 2026-02-22T10:24:08.605Z
Learning: In `packages/web/src/Locanara.ts`, the standalone web SDK targets Chrome 138+ which uses delta mode for `promptStreaming`, so cumulative-to-delta normalization is not required. The expo module (`libraries/expo-ondevice-ai`) has auto-detection as a safety net for broader version support.
Comment on lines
+209
to
+213
| export interface NitroTextStreamChunk { | ||
| delta: string; | ||
| accumulated: string; | ||
| isFinal: boolean; | ||
| } |
There was a problem hiding this comment.
The Nitro streaming contract cannot demultiplex concurrent streams.
NitroTextStreamChunk has no request/stream identifier, and each feature only exposes a global add/remove listener pair. Two in-flight streams of the same feature will be indistinguishable in JS, so chunks can fan out to the wrong consumer. Add a stream ID to the payload or explicitly enforce one active stream per feature.
Also applies to: 301-332
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@libraries/react-native-ondevice-ai/src/specs/OndeviceAi.nitro.ts` around
lines 209 - 213, The current streaming contract (NitroTextStreamChunk) lacks a
stream identifier so concurrent streams cannot be demultiplexed; update the
contract and listener API to include a unique streamId on NitroTextStreamChunk
(and any equivalent chunk interfaces referenced around the 301-332 region), and
propagate that id through the global add/remove listener pair (or alternatively
change the API to enforce a single active stream per feature by preventing
addListener if a stream is already active). Specifically modify the
NitroTextStreamChunk interface to include streamId and update the corresponding
addListener/removeListener handlers and any emitters/dispatchers to attach and
check streamId so JS consumers can route chunks to the correct in-flight stream.
Member
Author
There was a problem hiding this comment.
Acknowledged. Current design assumes sequential streaming calls. Concurrent stream demultiplexing is a future enhancement if needed.
There was a problem hiding this comment.
@hyochan, understood — thanks for the clarification! Noting this design decision for future reviews.
(´• ω •`) 🐇
✏️ Learnings added
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 20
File: libraries/react-native-ondevice-ai/src/specs/OndeviceAi.nitro.ts:209-213
Timestamp: 2026-03-21T09:12:15.012Z
Learning: In `libraries/react-native-ondevice-ai/src/specs/OndeviceAi.nitro.ts`, the streaming contract (`NitroTextStreamChunk` and per-feature add/remove listener pairs for `summarizeStreaming`, `translateStreaming`, `rewriteStreaming`) intentionally assumes sequential streaming calls. Concurrent stream demultiplexing (e.g., adding a `streamId` to the chunk) is deferred as a future enhancement and should not be flagged as a blocking issue in code review.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
🧠 Learnings used
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-23T14:32:41.094Z
Learning: In `packages/web/src/Locanara.ts`, the SDK's `chatStreaming` method internally handles both cumulative and delta modes from Chrome's `promptStreaming` API, always yielding delta chunks to consumers. This normalization happens at the SDK level, so example apps and other consumers can safely use simple concatenation (`response += chunk`) without worrying about the streaming mode.
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/example/main.ts:660-674
Timestamp: 2026-02-22T10:24:15.711Z
Learning: Chrome 138+ uses delta mode for promptStreaming API, where each chunk contains only the new text (delta) rather than the cumulative response. This means simple concatenation (`response += chunk`) is the correct approach when targeting Chrome 138+ as the minimum requirement.
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: Start Nitro Module API changes from the spec file (`src/specs/OndeviceAi.nitro.ts`) before modifying native implementations
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: Applies to libraries/react-native-ondevice-ai/src/specs/*.nitro.ts : Nitro union types must have 2 or more values
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts:28-34
Timestamp: 2026-02-23T14:50:03.100Z
Learning: In `libraries/expo-ondevice-ai/src/ExpoOndeviceAiModule.web.ts`, the cached Chrome Built-in AI instances (cachedSummarizer, cachedLanguageModel, cachedRewriter, cachedWriter, cachedTranslators) intentionally use `any` type instead of `types/dom-chromium-ai` because Chrome Built-in AI APIs are experimental and changing rapidly. Using `any` with runtime checks is more practical than depending on external type packages that may lag behind API changes.
<!-- [/add_learning]
Learnt from: hyochan
Repo: hyodotdev/locanara PR: 9
File: packages/web/src/Locanara.ts:576-592
Timestamp: 2026-02-22T10:24:08.605Z
Learning: In `packages/web/src/Locanara.ts`, the standalone web SDK targets Chrome 138+ which uses delta mode for `promptStreaming`, so cumulative-to-delta normalization is not required. The expo module (`libraries/expo-ondevice-ai`) has auto-detection as a safety net for broader version support.
Learnt from: CR
Repo: hyodotdev/locanara PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-02-26T20:59:28.030Z
Learning: Update both iOS (`ios/HybridOndeviceAi.swift`) and Android (`android/.../HybridOndeviceAi.kt`) native implementations when modifying Nitro Module API
…native layer - Run nitrogen to regenerate Nitro bindings from updated spec - Add summarizeStreaming/translateStreaming/rewriteStreaming to HybridOndeviceAi.kt and HybridOndeviceAi.swift - Add describeImage stub (Web SDK only) to both native implementations - Update mock and add 11 new tests covering all new APIs
There was a problem hiding this comment.
Pull request overview
This PR aims to bring wrapper-library API parity with the Web SDK by exposing web-unique capabilities (streaming summarize/translate/rewrite and describeImage) and adding supporting types/spec definitions so React Native and Expo consumers can call these APIs consistently.
Changes:
- Added shared streaming chunk types and
*Streamingoption types to wrapper public TypeScript APIs. - Extended the React Native Nitro spec to include
summarizeStreaming/translateStreaming/rewriteStreaminglistener-based streaming APIs anddescribeImage. - Committed new Nitrogen-generated iOS Swift bridge artifacts for new union/struct types.
Reviewed changes
Copilot reviewed 10 out of 216 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| libraries/react-native-ondevice-ai/src/types.ts | Adds public TS types for streaming chunks/options and describeImage request/response types. |
| libraries/react-native-ondevice-ai/src/specs/OndeviceAi.nitro.ts | Extends the Nitro native contract with streaming variants, stream listeners, and describeImage. |
| libraries/expo-ondevice-ai/src/types.ts | Adds Expo public TS types for streaming chunks/options and describeImage types. |
| libraries/react-native-ondevice-ai/nitrogen/generated/ios/swift/* | Adds generated Swift bridge types needed by the updated Nitro spec. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #12
Exposes Web SDK unique APIs (summarizeStreaming, translateStreaming, rewriteStreaming, describeImage) in expo-ondevice-ai and react-native-ondevice-ai. Adds API_PARITY.md documenting cross-platform API availability including Chrome-only features.
Summary by CodeRabbit
New Features
Documentation