add support more embedding models

[alyahmedaly]

I wanted to try Alibaba-NLP/gte-large-en-v1.5 and other models to see if I will get better results

[runonthespot]

Currently this project uses fastembed-rs which limits options to models supported (without contributing to that project.) so unlikely to fix this in the interim. If you like we could do something like enable calls to external embedding APIs on HF?

[abab-dev]

Please add embedding API support ( Jina, Voyager).
Potato pc owners will benefit greatly.

[neodon]

I would make use of a configurable embedding API. Probably OpenAI-compatible would give the widest coverage for a first step. The vector length would need to be configurable as well as the max chunk size.

A couple of examples:

One model I've used for embeddings is google/gemini-embedding-001 via OpenRouter. It has a max 2048 context and flexible vector length (I use 1536).

For GPU-accelerated local embeddings I use qwen3-embedding-4b via LM Studio (OpenAI compatible API). I have the context set at 1024 based on another indexing tool I used, but my system can handle 2048. The vector length for this model is fixed at 2560.

[runonthespot]

Status: between this issue and #101, there's clearly demand for external embedding APIs (OpenAI-compatible, Jina, Voyager, Gemini). We're going to tackle this in two steps:

1. Embedder trait redesign (in our backlog as a med-priority refactor) — the current ck-embed::Embedder trait is sync and ModelConfig has no place for base URL, auth env var, batch size, retry policy, or vector-length validation. This needs to land first.
2. OpenAI-compatible provider (some WIP already exists locally) — once the trait can hold those concerns, this plugs in cleanly. Jina/Voyager/Gemini follow the same template.

Tracking the trait work internally; will link back when there's a PR.