Ask AI on docs doesn't work

[PatrykKuniczak]

Describe the bug

After very first question:

After second:

Reference:
#2037

Reproduction

1. Visit docs
2. Click "Ask AI" button on the right bottom
3. Type anything and submit

Validations

• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[smashedr]

FWIW: These errors are standard with Gemini free tier, which is what I believe the plugin works with.

[PatrykKuniczak]

@smashedr Each probably you're all right, but maybe @aklinker1 Are able to solve it in some way

[smashedr]

Both these errors are usually transient depending on the Gemini server workload (503) and the WXT website workload (429).

In both cases the solution is usually to try again later. You would need a paid API key to prevent these errors.

That being said, Gemini logs all the usage and errors, so the key owner can validate all this.

[aklinker1]

I could pay... But people already have their own AI setups, and I don't think this needs to handle any real load. Just people popping in for the first time or an occasional question.

We already provide context files with the docs (though it ~700,000 tokens last I looked), and have integrations with context7. Newer models also have started to incorporate WXT's docs into their knowledge base, though it's hit or miss.

I can retry the 503 errors for sure, and try to compress the context more so we hit less 429s. I'm pretty sure we're hitting the tokens per minute rate limit, due to the big context, I'll double check later.

[aklinker1]

The other option is to use a different AI search provider/chatbot, that was discussed in a different issue maybe?

[smashedr]

The other option is to use a different AI search provider/chatbot, that was discussed in a different issue maybe?

Out of curiosity, what other provider/chatbot are you considering using?

[aklinker1]

Algolia has an integration with VitePress, that would be the easiest, but I've had bad experiences with algolia just not indexing things correctly.

There's another I can't remember, but you need to request access and I've never heard back from them.

[aklinker1]

Another approach would be to make vitepress-knowledge smarter, and give it tools for searching the docs and only loading relevant ones, like how AI agents wrote code.

That would probably be the smart thing to do, keep it free and easy to host, use less tokens.

[smashedr]

Another approach would be to make vitepress-knowledge smarter, and give it tools for searching the docs and only loading relevant ones, like how AI agents wrote code.
That would probably be the smart thing to do, keep it free and easy to host, use less tokens.

That would be, very interesting to see...

I have quite a few VitePress sites and would also like the ability to use an arbitrary provider/model from claude/openai/gemini.

I recently setup a simple test action using the ai-sdk.dev which provides a single typescript api for using all of these providers. The user just provides the model and API Key environment variable.
Reference: https://github.com/smashedr/ai-issue-action

[smashedr]

Not to hijack this issue, but I did setup a quick test using the ai-sdk chat and was able to get a fully functional VitePress Chat Client and Proxy API Server, both in less than 30 lines of code each. Pretty impressive SDK.

pwsh-20260516-191251630.mp4

[smashedr]

@aklinker1 do you want to, and is there somewhere, we can collaborate on this?

I packaged the secure proxy server into a docker container: https://github.com/smashedr/ai-chat-server
I packaged the live streaming client into a VitePress plugin: https://github.com/smashedr/vitepress-chat

The server holds the API key and live streams the API results to the client.
The client generates instructions from the docs and adds the chat button/box.

There is a live demo on the client docs: https://smashedr.github.io/vitepress-chat/