Add ChromeAI

[jtpio]

Towards #24

Experimenting with built-in (on device) AI, such as Chrome AI: https://developer.chrome.com/docs/ai/built-in

langchain.js supports it as a text completion model, still experimental for now: https://js.langchain.com/docs/integrations/llms/chrome_ai/

This requires:

• using Google Chrome
• enabling some flags
• downloading the model (likely Gemini Nano), which should happen automatically on first use

One of the main advantages (omitting the browser limitation and required flag) is the zero setup needed (no API key).

jupyterlite-ai-chrome-ai.mp4

[jtpio]

OK looks like we could stop here for now as this seems to be working fine, provided that requirements are met (Chrome + flags enabled).

[jtpio]

Pushed one more commit to not discard completions that may be formatted as code blocks.

[brichet]

Thanks @jtpio.

I had to enable more flags to download the model locally. These steps where describes in https://chromeai.org/ (the link you mentioned in the README. I wonder if we should add these steps directly in the README.

Necessary Experimental Flags

1. chrome://flags/#prompt-api-for-gemini-nano
   Select 'Enabled'

2. chrome://flags/#optimization-guide-on-device-model
   Select 'Enabled BypassPrefRequirement'

3. chrome://components
   Click 'Check for Update' on Optimization Guide On Device Model to download the model. If you don't see Optimization Guide, ensure you have set the flags correctly above, relaunch your browser, and refresh the page.

[brichet]

Also on my side this is really slow. Do you know if there are some set up to make it faster ?

[brichet]

On a side note, I realized from you demo that the dates of messages are wrong.
I have the same locally, this probably need to be fixed in jupyter-chat.

[brichet]

Do you know if there a way to detect the browser, and have this provider available for chrome only ?

[jtpio]

Maybe just checking for window.ai availability could be enough?

[jtpio]

Since we specify the name of the model as part of the settings definition:

"enum": ["None", "Anthropic", "ChromeAI", "MistralAI"]

Not sure there could be a nice way to dynamically enable / disable support for ChromeAI, based on window.ai? Unless we manually check for "ChromeAI" and remove it from the settings if window.ai === undefined.

[brichet]

Not sure there could be a nice way to dynamically enable / disable support for ChromeAI, based on window.ai?

Yes, that settings management is already hazardous, don't no neither if we want to add more manual changes.
Maybe the way to go will be to add a dedicated configuration panel that doesn't rely on settings...

[jtpio]

Maybe the way to go will be to add a dedicated configuration panel that doesn't rely on settings...

Agreed, sounds like it would give more flexibility 👍

[brichet]

I opened #35

[jtpio]

Also on my side this is really slow. Do you know if there are some set up to make it faster ?

Yeah it may be quite slow on some devices indeed.

We could mention the same list of flags as on https://chromeai.org/ for completeness, and also leave a note the performance may not be great on some devices. Not sure there is anything more to do about this at the moment.

[brichet]

Thanks, LGTM