RAG Query Assistant

RAG Query Assistant is a retrieval-augmented generation backend for uploading documents, indexing them locally, and answering questions with citations. Answers are grounded in retrieved evidence, and the system returns an insufficient-evidence result instead of guessing when support is weak.

What it does

• Upload PDF, Markdown, and plain text files.
• Parse documents into chunks, embed them, and store them in a local SQLite index.
• Run SQLite-backed vector search over chunk embeddings for retrieval.
• Track ingestion jobs with queued, processing, completed, and failed states.
• Ask questions against all indexed documents or a selected document.
• Return grounded answers with citations, retrieved chunks, and support diagnostics.
• Retry retrieval with query reformulation when initial search quality is poor.
• Reject unsupported answers with a structured insufficient-evidence response.

Architecture

• Backend: FastAPI
• Ingestion: file loading, chunking, embedding, and index persistence
• Retrieval: embedding-based chunk search with optional document scoping
• Answering: grounded prompt building, LLM generation, citation resolution, and support verification
• Storage: SQLite for documents, chunks, and ingestion jobs

API Endpoints

• GET /health - basic service health check.
• POST /v1/ingest/upload - upload a document for ingestion.
• GET /v1/ingest/jobs/{job_id} - check ingestion job status.
• GET /v1/ingest/documents/{document_id} - fetch document metadata.
• GET /v1/ingest/documents/{document_id}/chunks - fetch indexed chunks for a document.
• POST /v1/ask/ask - ask a grounded question.

Configuration

The backend reads settings from backend/.env via Pydantic settings. The table below covers the main tuning knobs and their defaults.

Setting	Default	Purpose
APP_NAME	RAG Query Assistant	FastAPI app title.
UPLOAD_DIR	data/uploads	Directory for uploaded source files.
SQLITE_PATH	data/index/index.sqlite3	Local index database path.
MAX_FILE_SIZE_MB	25	Maximum upload size.
CHUNK_SIZE_CHARS	1200	Chunk size during document splitting.
CHUNK_OVERLAP_CHARS	200	Overlap between adjacent chunks.
EMBEDDING_MODEL	embeddinggemma	Embedding model name.
EMBEDDING_BASE_URL	http://localhost:11434/api	Embedding API base URL.
OLLAMA_BASE_URL	https://ollama.com/api	LLM API base URL.
OLLAMA_API_KEY	empty	API key for the LLM provider.
LLM_MODEL	gemma4:31b	Model used for answer generation.
EMBEDDING_BATCH_SIZE	32	Batch size for embedding requests.
EMBEDDING_TIMEOUT_SECONDS	60	Embedding request timeout.
LLM_TIMEOUT_SECONDS	60	LLM request timeout.
ANSWERING_DEFAULT_TOP_K	8	Default number of chunks to retrieve.
ANSWERING_MAX_TOP_K	30	Hard cap on retrieved chunks.
ANSWERING_CANDIDATE_LIMIT	1000	Retrieval candidate limit.
ANSWERING_CORRECTIVE_ENABLED	True	Enables corrective retrieval retries.
ANSWERING_CORRECTIVE_MAX_RETRIES	5	Maximum rewritten query attempts.
ANSWERING_MIN_INITIAL_MAX_SCORE	-0.15	Threshold for weak first-pass retrieval.
ANSWERING_MIN_INITIAL_AVG_SCORE	-0.35	Average-score threshold for weak retrieval.
ANSWERING_MIN_INITIAL_UNIQUE_CHUNKS	2	Minimum unique chunks before retrying.
ANSWERING_QUERY_REWRITER_KEYWORD_MIN_LENGTH	3	Minimum keyword length for rewrites.
ANSWERING_QUERY_REWRITER_KEYWORD_LIMIT	8	Maximum keyword count in a rewritten query.
ANSWERING_QUERY_REWRITER_LLM_FALLBACK_LIMIT	1	LLM-generated rewrite fallback count.
VERIFIER_MIN_SUPPORT_SCORE	0.25	Minimum support score for a valid answer.
VERIFIER_MIN_OVERLAP_RATIO	0.20	Minimum answer/evidence token overlap.
VERIFIER_MIN_CITATION_COVERAGE	0.10	Minimum citation coverage across retrieved evidence.
VERIFIER_MAX_UNSUPPORTED_TERMS	8	Maximum unsupported terms before rejection.

Local Setup

Prerequisites:

• Python 3.12+
• An Ollama-compatible LLM endpoint and embedding endpoint, or equivalent providers configured in .env

Backend:

1. Create the Conda environment from environment.yml:

   conda env create -f environment.yml

2. Activate the environment:

   conda activate rag-query-assistant

3. Copy backend/.env.example to backend/.env and update endpoint/model values as needed.

4. Start the backend API from the project root:

   cd backend
   uvicorn app.main:app --port 8000 --reload

5. Open Swagger UI for interactive API testing:

   • http://127.0.0.1:8000/docs
   • http://127.0.0.1:8000/redoc

6. Test the APIs in Swagger:

   • Run GET /health first to confirm the service is up.
   • Use POST /v1/ingest/upload to upload a document.
   • Use POST /v1/ask/ask to submit a grounded question.
   • Check ingestion progress with GET /v1/ingest/jobs/{job_id}.