v9.3.0 - the finale

Release Notes

This release brings a major refresh of the model lineup, a rebuilt Ask Jeeves assistant, a new MiniMax backend (credit to @octo-patch), and a large round of dependency upgrades and stability fixes. It's likely the last release before upgrading to transformers 5+, which will enable additional features but may deprecate certain models.

🤖 Chat Models

New models

Model	Details
Phi 4 – 14b	➕ Excels at not hallucinating, essential for RAG
Gemma 3 – 4b	➕ Excels at not hallucinating, essential for RAG
Gemma 3 – 12b	➕ Excels at not hallucinating, essential for RAG
Granite – 3b	🔄 Upgraded from Granite 3.3 → 4.1
Granite – 8b	🔄 Upgraded from Granite 3.3 → 4.1
LiquidAI – 1.2b	🔄 Updated to LFM2.5-1.2B-Instruct

• Qwen 3 models now run in non-thinking mode for direct, concise answers. Research shows that "thinking" appreciably reduces quality for purposes of RAG.

Removed Models
Deepseek R1 – 8b
GLM4-Z1 – 9b
GLM4-Z1 – 32b
Seed Coder – 8b
EXAONE
Phi 4 Mini – 4b
Qwen 3 – 4b (Thinking)
Qwen 3 – 32b

• All removed models are either too heavy for too low quality compared to newer models or require "thinking" (or both).

🧬 Embedding Models

New models

Model	Details
harrier-oss-v1-270m	➕ Great generalist and also excels at multilingual (~94 languages)
harrier-oss-v1-0.6b	➕ Great generalist and also excels at multilingual (~94 languages)
Octen-Embedding-0.6B	➕ Great generalist and also excels at legal, medical, finance, and code.
Octen-Embedding-4B	➕Great generalist and also excels at legal, medical, finance, and code.
Octen-Embedding-8B	➕ Great generalist and also excels at legal, medical, finance, and code.
modernbert-embed-base_finetune_512	➕ Excels at legal.
modernbert-embed-base_finetune_8192	➕ Excels at legal
EmbeddingGemma	➕ Google, 300M — Strong and efficient generalist

Removed Model
Granite-30m-English
Granite-125m-English
inf-retriever-v1-1.5b
inf-retriever-v1-7b
arctic-embed-m-v2.0
arctic-embed-l-v2.0

• The removed models were essentially superseded by smaller models with at-par or better results.

👁️ Vision Models

New / updated models

Model	Details
Liquid-VL – 480M	➕ New — LFM2-VL family, fast and low-VRAM
Liquid-VL – 1.6B	➕ New — LFM2-VL family, fast and low-VRAM
Liquid-VL – 3B	➕ New — LFM2-VL family, fast and low-VRAM
Qwen3-VL – 2B	➕ Qwen VL lineup (Qwen3-VL generation)
Qwen2.5-VL – 3B	➕ Qwen VL lineup
Qwen3-VL – 4B	➕ Qwen VL lineup (Qwen3-VL generation)
Qwen2.5-VL – 7B	➕ Qwen VL lineup
Granite Vision 3.2 – 2B	🔄 Updated to 3.2 (2B)
InternVL3 – 1B	🔄 Now loads from the official -HF repo
InternVL3 – 2B	🔄 Now loads from the official -HF repo
InternVL3 – 8B	🔄 Now loads from the official -HF repo

Performance

• Qwen-VL patch-embed Conv3d is now swapped for a mathematically identical matmul, sidestepping a slow cuDNN kernel that could cost ~12–15s per image on some GPUs.

Removed Model
Ovis2
Florence-2
GLM-4.1V-9B-Thinking
InternVL3 – 14b
Molmo-D-0924

• All superseded by newer models that are lighter and/or produce better results.

🔊 Text-to-Speech

• Added Kyutai (GPU) and Kyutai Pocket (CPU) backends.
• Chatterbox can now run on CPU or GPU.

☁️ MiniMax Backend

• Added MiniMax-M3 chat backend.

🛎️ Ask Jeeves

• Rebuilt from the ground up — Jeeves now runs the same lightweight, unquantized CHAT_MODELS (LiquidAI and Qwen 3 small models) instead of the old CTranslate2 int8 models.
• New Sources panel — a collapsible "Show sources" view lists the retrieved user-guide chunks with clickable links and similarity scores.
• Refined the butler persona for clearer, more in-character answers.
• Added an easter egg...

🖥️ UI & Features

• Local chat models now work on CPU — the GPU-only restriction has been removed.
• The GPU comparison chart plus backup/restore remain; the chat/vision/vector model comparison charts have been retired.
• Expanded user manual coverage.

🐛 Stability & Bug Fixes

• Pinned numpy to 2.3.4 to avoid a heap-corruption regression in numpy 2.4.x during large TileDB builds, and added a np.in1d → np.isin shim for tiledb-vector-search.
• Generation threads now propagate errors instead of failing silently; metrics-bar collection moved to an interruptible loop.
• Centralized subprocess output draining with proper timeout handling so stuck stages can't hang the app.
• Embedding pipeline now safely drops and reports chunks that fail tokenization rather than crashing.
• TileDB write path optimized (consecutive-metadata dedup, vectorized ID strings, structured-array views) to reduce memory churn at scale, with cleanup on metadata-DB failure.
• Transcription and voice-recording now use proper Qt worker threads; OCR pre-check uses threads instead of a spawn pool.
• Assorted fixes: OpenAI/LM Studio streaming error handling, documentation-scraper path matching, TTS normalization guard, vision test text wrapping, model-download index handling, and a proper __main__ guard, etc.

🧰 Dependencies & Build

• Broad dependency refresh across the stack (PyTorch CUDA wheels, transformers, sentence-transformers, PySide6, pymupdf, and many more), with carefully chosen version caps documented inline to keep the environment consistent.
• Added tools/build_user_manual_db.py to fully automate building, zipping, and deploying the Ask Jeeves knowledge-base database.
• setup_windows.py now enables ANSI color output in the Windows console.

v9.2.0

🎨 Theming Overhaul

• Replaced 25+ individual CSS files with a single template-based theming system using variable substitution

🤖 Chat Backend Improvements

• New unified Chat Backend Settings dialog (File menu) with tabs for ChatGPT, LM Studio, MiniMax, and Kobold
• Added support for OpenAI's GPT-5.4 and GPT-5.5 model families with verbosity and reasoning effort controls
• Live API cost panel displays input/cached/output pricing per model
• Migrated ChatGPT integration to OpenAI's Responses API for streaming
• Improved thinking-tag stripping in LM Studio chat to handle tags split across stream chunks

📚 Database & Document Processing

• Added support for .htm files alongside existing .html ingestion
• Folder selection now prompts whether to include subdirectory files
• New Cancel button for in-progress database creation with automatic cleanup of partial files
• Enhanced info bar showing queued files, chunk size, overlap, and embedding precision in real-time
• Symlink creation now handles filename collisions via hash-based disambiguation

🕷️ Documentation Scraper

• New Resume capability for interrupted scrapes — pick up where you left off instead of starting over
• Added Mintlify scraper for llms.txt-based documentation sites (including Model Context Protocol docs)
• Rate-limit detection (HTTP 429) automatically pauses scrapes and persists state for later resumption
• Added scrapers for curl_cffi, emoji, lxml, onnx, pyparsing, and uv documentation

🔧 UI Polish & Fixes

• Reorganized Database Query and Database Creation tabs with cleaner grid layouts
• Vector Models tab now allows re-downloading already-downloaded models with confirmation
• Voice recorder button now shows "Stop Recording" while active
• Submit Question button now styled green and set as default
• Removed unused chat history file writes and Jeeves context spinner
• Fixed search term filter tooltip (now correctly describes inclusion, not exclusion)

🐛 Bug Fixes

• Fixed TileDB search returning misaligned results when vector IDs don't match query order
• Fixed transcription and OCR worker processes to properly surface errors via exit codes
• Fixed PDF and audio file handles to use context managers, preventing resource leaks
• Fixed checkpoint saves to use atomic os.replace with retry on Windows permission errors
• Removed dependency on LangChain in favor of internal Document class

v9.1.0

🎉 Release Notes

✨ New Features

🎙️ Kyutai Pocket TTS Backend

• Added a brand-new CPU-only Kyutai Pocket TTS backend — no GPU required
• 21 voices available (alba, anna, azelma, charles, cosette, eve, fantine, george, jane, marius, and more)
• Optional int8 quantization toggle for ~48% less RAM and ~27% faster inference with no quality loss

🔄 Concurrent Documentation Scraping

• Run up to 6 documentation scrapes simultaneously instead of one at a time
• New per-scrape UI with individual cancel buttons, "Open Folder" buttons, and live page counts

📊 Vision Model Comparison Dialog

• Replaced the old progress bar with a rich, per-model status list
• Visual indicators for each model's state: ⏸ pending, ⏳ running, ✅ success, ❌ failed

🗣️ WhisperSpeech Speaker Selection

• New Speaker dropdown for WhisperSpeech (default, classic, voice_b)

🧼 Smarter Scraper Content Extraction

• New post-extraction cruft stripper removes embedded clutter from scraped pages, producing cleaner, more vector-DB-friendly HTML
• Strips Sphinx TOC trees, prev/next nav bars, sidebars, scripts, styles, SVGs, and footers from inside the main content
• Removes theme-specific noise: MkDocs Material "Edit this page", Pydata "On this page" sidebars, Pydantic.dev feedback widgets, PyMuPDF version footers, SciPy "Try it in your browser" sandboxes, and more

🚀 Improvements

Kyutai TTS Overhaul

• Added model size selector: choose between 1.6B (EN+FR, ~4.2GB VRAM) and 0.75B (EN, ~2GB VRAM)
• Quality presets replaced with proper per-model n_q configuration
• Smarter checkpoint loading that reads weight filenames from the model's own config

WhisperSpeech Model Lineup

• Added new models: s2a-q4-small, s2a-v1.1-small, t2s-small, t2s-fast-small, and t2s-fast-medium
• Migrated from whisperspeech to whisperspeech2 with optimize=True
• Smart CUDA graph auto-disable for known-incompatible models

Web Scraper Engine

• Switched from aiohttp to curl_cffi with Chrome impersonation for better compatibility with bot-protected docs sites
• Cleaner error handling and retry flow
• Fixed scrape output directory to use PROJECT_ROOT instead of a module-relative path

HTML Parsing

• Switched BeautifulSoup parser from html.parser to lxml for faster, more robust document and email parsing

📚 Documentation Database

• Added/updated: bitsandbytes 0.48.2, datasets 4.3.0, deepdiff 8.6.1, Diffusers 0.35.0, Huggingface Hub 0.36.0, Optimum (main), Optimum ONNX (main), Safetensors (main), SciPy 1.16.2, sounddevice 0.5.3, SpeechBrain (latest), Timm 1.0.20, tokenizers 0.22.1, torch 2.9, Torchaudio 2.9, Torchvision 0.24, Transformers 4.57.5
• Scraper-class refinements: corrected scraper assignments for many entries (cffi, chardet, click, coloredlogs, cryptography, CTranslate2, cycler, Deprecated, distro, fonttools, pi-heif, PyOpenGL, python-dateutil, Rich, scikit-learn, Six, sniffio, SoundFile, SQLAlchemy, tenacity, Tile DB, tiledb-cloud, webencodings, xlrd, and others) for cleaner, more accurate extraction
• Removed (cleanup of unused docs): Argcomplete, Black, CLIO, cuDF, isort, jiwer, kdenlive, Langchain, Librosa, llama-cpp-python, Loguru, lxml-html-clean, Numexpr, NumPy 1.26, Pandoc, PyInstaller, Pywin32, Soxr, SpaCy, tblib, Transformers.js, uv, xFormers, and several older PyTorch/Torchvision/Torchaudio versions

🐛 Bug Fixes

• OCR: Fixed AttributeError in HocrTransform when tesserocr's hOCR output omits the ocr_page div or its bbox — width/height are now always set explicitly
• TTS GPU check: kyutaipocket is now correctly recognized as a CPU backend and bypasses the GPU-required warning
• ChatTTS: Removed lazy-import workaround now that the dependency is updated
• Vision models: Switched Qwen2.5-VL and GLM-4V to AutoModelForImageTextToText with device_map="auto" for more reliable loading

🧹 Cleanup

• README trimmed of redundant build-tools screenshots and instructions
• Removed unused charset-normalizer fallback encoding logic from the scraper (curl_cffi handles this natively)
• Removed obsolete quality-mapping dict from the Kyutai backend

v9.0.1

• Update CPU-only models list.

v9.0.0

🚀 VectorDB-Plugin v9.0.0 — Major Architecture Overhaul

This release represents a substantial refactoring of the codebase, introduction of new features, and significant improvements to scalability and reliability for large-scale database creation.

📁 Project Restructure

The flat src/ directory has been reorganized into a proper package structure for better maintainability:

• core/ — Configuration, constants, utilities, and initialization
• db/ — Database creation, querying, and document processing
• chat/ — All chat backends (LM Studio, Kobold, OpenAI, Local Model, Jeeves, MiniMax)
• gui/ — All GUI components organized into logical subdirectories (tabs_databases/, tabs_models/, tabs_settings/, tabs_tools/)
• modules/ — Processing modules (TTS, OCR, transcription, image processing, scraping)
• charts/, tools/ — Helper utilities

⚙️ The New Subprocess-Isolated Pipeline (⭐ Headline Feature)

Database creation has been completely re-architected around a multi-stage, subprocess-isolated pipeline designed to handle databases of effectively any size — including million-chunk corpora that would crash the previous monolithic process. Every stage runs in its own fresh Python interpreter, with checkpoint-and-resume logic so a crash in one stage (or one worker within a stage) doesn't lose the work that came before it.

🧱 Why subprocesses (not just multiprocessing)?

On Windows + PySide6, multiprocessing with spawn inherits DLL state from the GUI process (TileDB, CUDA, torch). At scale this caused access violations (0xC0000005) and silent hangs. The new pipeline uses subprocess.Popen with a fresh sys.executable, which gives each stage:

• A clean DLL state (TileDB DLLs are explicitly preloaded per-process)
• Its own heap (no fragmentation carryover from the parent)
• Memory that's guaranteed released when the stage exits (not just garbage-collected)
• Crash isolation — a segfault kills the child, not the GUI

The GUI thread streams the child's stdout line-by-line into the log window, so progress is visible in real time even though the work is happening in another process.

1️⃣ Stage 1: Document Extraction (stage_extract.py)

• Launched as an isolated subprocess
• Reads files from Docs_for_DB/ and dispatches to the right loader (PDF via PyMuPDF, DOCX, TXT, CSV, HTML, EML, MSG, XLS/XLSX, RTF, MD)
• Uses a ProcessPoolExecutor internally with THREADS_PER_PROCESS=4 so each extraction worker can parallelize across multiple files
• Up to 3 retry attempts at the stage level (EXTRACT_MAX_RETRIES) — if the whole stage crashes, the parent waits 3 seconds, runs gc.collect(), and re-launches
• Output: a single pickle file containing (content, metadata) tuples for every successfully loaded document

2️⃣ Stage 2: Document Splitting (stage_split.py)

This is where the wave-based parallelism starts. Splitting a million chunks in one process risks heap fragmentation, so the work is sharded:

• Documents are partitioned into worker batches of split_worker_batch_size (1,000–5,000 docs depending on the performance preset)
• Each worker batch runs in its own subprocess — so if one worker crashes on a malformed doc, only that batch is affected
• Workers are launched in parallel waves of split_max_parallel_workers at a time (1 to N depending on preset). Each wave runs concurrently via ThreadPoolExecutor, with each thread shepherding its own subprocess
• Per-worker retries: each worker gets up to SPLIT_MAX_WORKER_RETRIES=3 attempts before being declared failed
• Stage-level retries: the entire splitting stage has up to SPLIT_MAX_RETRIES=5 attempts on top of that
• Checkpointing every 5 workers: progress is atomically written to a .tmp file then renamed, so a crash mid-stage resumes from the last checkpoint instead of starting over
• Sequential mode kicks in for small jobs (≤5,000 docs) since spinning up parallel subprocesses isn't worth the overhead at that scale

3️⃣ Stage 3: Tokenization (stage_tokenize.py)

The most sophisticated stage, with three different batch-size knobs:

• Tokenization runs in its own subprocess (so the embedding model isn't loaded in the same interpreter as the tokenizer — keeping VRAM clean before the forward pass)
• Workers receive worker_batch_size=60,000 texts each (WORKER_BATCH_SIZE)
• Inside each worker, texts are tokenized in mini-batches of TOKENIZE_BATCH_SIZE=100 to keep memory pressure low
• Workers run in parallel waves of tokenize_max_parallel_workers (preset-driven)
• Length-sorted batching: after tokenization, sequences are sorted by length before padding into encode batches. This dramatically reduces padding waste — typical efficiency improvements of 30%+ since short sequences end up batched with other short sequences instead of being padded to match a single long outlier
• Encode batch size is model-aware: bge-small uses 100, bge-large uses 50, Qwen3-Embedding-0.6B uses 10, Qwen3-Embedding-4B uses 5. Falls back to a VRAM-based heuristic (int(vram_gb * 4), clamped to [10, 256]) for unknown models
• Checkpoint every 5 workers — the checkpoint records next_text_index so a crashed retry resumes from exactly the right offset, not from the start of the worker batch
• Up to 5 stage-level retries (TOKENIZE_MAX_RETRIES) plus 3 per-worker retries — and crucially, partial progress from crashed attempts is recovered from the checkpoint file before retrying, so even a worker that died at 90% completion contributes its 90% to the final result
• Tracks padding efficiency (real tokens vs pad tokens) and reports it in the log

4️⃣ Stage 4: Embedding (in-process, with isolation between batches)

• The embedding model is loaded only in the main creation subprocess (not in the tokenize workers — they only produce token IDs)
• Pre-padded batches arrive from the tokenize stage and are run through the model's forward pass
• Memory hygiene: gc.collect() every 50 batches, torch.cuda.empty_cache() to prevent VRAM creep
• Batches are unsorted back to original document order via seq_indices before being written to TileDB

5️⃣ Stage 5: TileDB Write

• Vectors are written to TileDB in batches of TILEDB_WRITE_BATCH_SIZE=100,000 to keep peak memory bounded
• Random uint64 IDs are generated per-batch using numpy.random.default_rng().integers() — vectorized in C, avoiding the Python int allocation storm that triggered OverflowError + access violations on million-chunk databases in earlier versions
• JSON metadata serialization uses orjson (Rust-based, ~10x faster than stdlib) when available — again, avoiding heap fragmentation under massive serialization loops
• Final consolidation + vacuum, then FLAT index creation via tiledb.vector_search.ingest()

🎚️ How presets tune the pipeline

The new Pipeline Performance dropdown (Database Create settings tab) controls every parallelism knob in one place:

Preset	Ingest Threads	Ingest Procs	Split Workers	Tokenize Workers	Split Worker Batch
🐢 Minimal	1	1	1	1	5,000
🚶 Low	4	2	2	2	3,000
🏃 Normal	up to 8	up to 4	up to 4	up to 4	2,000
🚴 High	up to 16	up to 8	up to 8	up to 8	2,000
🚀 Maximum	all cores	all cores	all cores	all cores	1,000

Smaller worker batches at higher tiers means more checkpoints and finer-grained retry recovery; larger worker batches at lower tiers means less subprocess-launch overhead.

🔧 New Configuration System

• Replaced ad-hoc YAML reading with a Pydantic-based typed configuration (core/config.py)
• Validation, type coercion, and atomic writes via temp file + rename
• Thread-safe singleton accessor with get_config() / reload_config()

✨ New Features

🤖 MiniMax Chat Backend

• Added support for MiniMax-M2.7 and MiniMax-M2.7-highspeed models via OpenAI-compatible API
• New "MiniMax API Key" option in the File menu for credential management
• Full streaming support consistent with other chat backends

🛡️ Process & Resource Management

• New ProcessManager singleton (db/process_manager.py) tracks every spawned multiprocessing.Process and provides graceful, time-bounded cleanup on shutdown — prevents leaked Python child processes that hold open TileDB arrays, model weights, or CUDA contexts

v8.3 - python 3.13 support

Support python 3.13 and update dependencies, notably torch 2.9.0..

v8.2 - Gemma Embedding

Release Notes

Dependencies

• Updated dependencies
• Upgraded PyTorch → 2.8.0
• Upgraded CUDA → 12.8

Chat Models

Added / Updated	Removed
LiquidAI - 0.35b	MiniCPM4 - 0.5b
LiquidAI - 0.7b	Exaone - 2.4b
LiquidAI - 1.2b	Exaone - 7.8b
Qwen 3 - 4b (updated)	MiniCPM4 - 8b
	Exaone - 32b

Embedding Models

Added / Updated	Removed
embeddinggemma-300m	—

Vision Models

Added / Updated	Removed
InternVL3 - 1b (updated)	Ovis2 - 1b
Liquid-VL - 480M	Ovis2 - 2b
Liquid-VL - 1.6B	THUDM glm4v - 9b
GLM-4.1V-9B-Thinking	Molmo-D-0924 - 8b

Improvements

• Improved database creation process
• Added HuggingFace key handling for gated embeddings (e.g., embeddinggemma-300m)
• Revamped vision model settings

v8.1.0 - fixin to leave

This may be the last release for awhile. It attempts to fix some basic things that were preventing Qwen3 embedding models and a few others from working correctly.

v8.0.0 - Qwen3 Embeddings!

Version 8.0.0 changes (from 7.10.0):

Chat Models

• Added Deepseek R1 - excellent new thinking model

Embedding Models

• Added the current top-ranking embedding models Qwen3-Embedding-0.6B, Qwen3-Embedding-4B, and Qwen3-Embedding-8B
• Removed gte-Qwen2-1.5B-instruct - superseded by Qwen3 models

Text to Speech

• Added the exciting new chatterbox backend

Tools Tab

• Added a chart showing embedding model memory usage.

• Fixed scraping of libraries significantly and commented out TO DO to fix.
• Added Radeon RX 9060 series to GPU chart.

Ask Jeeves

• Updated Jeeves' knowledge to include any/all instructions to run the Kobold chat backend.
• Updated Jeeves' knowledge in general.

General Improvements

• Updated dependencies.
• Refactored database operations.
• Eliminated padding/truncation errors.

Installation

• Just follow the installation instructions on the github readme.

Upgrading from a Prior Version

INSTRUCTIONS FOR UPGRADING FROM A PRIOR VERSION

To upgrade from a prior version while still keeping your databases and models that were downloaded, perform the following steps:

• In your directory that contains all of the files currently...delete the Include, Lib, Scripts folders. (pertain to your virtual environment)
• Delete the pyvenv.cfg file. (pertains to your virtual environment)
• At this point, any and all files pertaining to your virtual environment are gone.
• Delete the Assets and CSS folders. (new ones will be provided in their entirety further below)
• Extract all the files from the .zip file for this release and go into the src folder.
• Copy all files and folders EXCEPT THE CONFIG.YAML FILE into the aforementioned folder where you virtual environment USED TO BE. If there are any prompts indicating that files will be overwritten, CLICK YES since we're only updating necessary files now.
• EXCEPT THE CONFIG.YAML FILE!
• EXCEPT THE CONFIG.YAML FILE!
• last time...EXCEPT THE CONFIG.YAML FILE!

Afterwards, open a command prompt and run the installation instructions on this repository's page; specifically:

python -m venv .

.\Scripts\activate

python setup_windows.py

7.10.0 - Ask Jeeves Again

Version 7.10.0 changes (from 7.9.0):

• Fixed Ask Jeeves and updated his knowledge.
• Stop text to speech within Jeeves midstream.
• Updated libraries to scrape.
• Misc.

Installation

• Just follow the installation instructions on the github readme.

Upgrading from a Prior Version

INSTRUCTIONS FOR UPGRADING FROM A PRIOR VERSION

To upgrade from a prior version while still keeping your databases and models that were downloaded, perform the following steps:

• In your directory that contains all of the files currently...delete the Include, Lib, Scripts folders. (pertain to your virtual environment)
• Delete the pyvenv.cfg file. (pertains to your virtual environment)
• At this point, any and all files pertaining to your virtual environment are gone.
• Delete the Assets and CSS folders. (new ones will be provided in their entirety further below)
• Extract all the files from the .zip file for this release and go into the src folder.
• Copy all files and folders EXCEPT THE CONFIG.YAML FILE into the aforementioned folder where you virtual environment USED TO BE. If there are any prompts indicating that files will be overwritten, CLICK YES since we're only updating necessary files now.
• EXCEPT THE CONFIG.YAML FILE!
• EXCEPT THE CONFIG.YAML FILE!
• Last time...EXCEPT THE CONFIG.YAML FILE!

Afterwards, open a command prompt and run the installation instructions on this repository's page; specifically:

python -m venv .

.\Scripts\activate

python setup_windows.py