Skip to content

RAG query pipeline: @rag-search retrieval endpoint + answer generation (#79) - #90

Merged
reekitconcept merged 2 commits into
feature-ai-ragfrom
kid/458/query-pipeline
Jul 23, 2026
Merged

RAG query pipeline: @rag-search retrieval endpoint + answer generation (#79)#90
reekitconcept merged 2 commits into
feature-ai-ragfrom
kid/458/query-pipeline

Conversation

@reekitconcept

Copy link
Copy Markdown
Member

Step 3 of the RAG MVP (ticket: distribution-kitconcept-intranet#458, part of #79): a question to @rag-search returns {answer, sources} end to end. Second commit is the removable RAG: TESTING: UI aid.

What's included

  • 3.1 Grounded prompt template: answer strictly from the provided context, decline when the answer is not in it, reply in the question's language; <think> blocks stripped
  • 3.2 Query pipeline (rag/pipeline.py): embed the question → {!knn} chunk retrieval with the same security/path/language pre-filters as the classic search → collapse chunks to parent documents → generate. RagResult with error codes (not_configured / embedding_failed / generation_failed / solr_unavailable); generation failure still exposes the retrieved sources; empty retrieval is not an error
  • 3.3 @rag-search restapi service on the pipeline (q required → 400, optional path_prefix/lang), replacing the TESTING draft endpoint
  • Robustness: raw socket errors from a dead Solr server map to solr_unavailable instead of a 500
  • Indexing fix (to the already-merged foundations): Image content is excluded from chunking — its only chunkable text is title/description, which polluted answer sources with photos; File stays included (its title/description locates real documents, and the post-MVP Tika body extraction builds on it)
  • Docs: Step 3 design decisions and known issues (shared test Solr core, topK source tail) in IMPLEMENTATION-79.md
  • TESTING commit: with the Use AI toggle on, the answer now renders above the classic result list instead of replacing it (the old routing displayed a misleading "No results")

Verification

Against the bundled German demo corpus (#89): golden questions 19/19 retrieval hits, 2/2 declines, no binary content cited as sources after the indexing fix; permission trimming holds (private pages absent for the plain member f.meier); RAG test suite 110 passed.

#79)

Step 3 of the RAG MVP (#458): a question to @Rag-Search returns
{answer, sources} end to end.

- 3.1 grounded prompt template: answer strictly from the provided
  context, decline when the answer is not there, reply in the
  question's language; <think> blocks stripped from the output
- 3.2 rag/pipeline.py: run_rag_search - embed the question, {!knn}
  chunk retrieval with the same security/path/language pre-filters as
  the classic search, collapse chunks to parent documents, generate;
  RagResult with error codes (not_configured/embedding_failed/
  generation_failed/solr_unavailable); generation failure still exposes
  the retrieved sources; empty retrieval is not an error
- 3.3 thin @Rag-Search restapi service on the pipeline (q required,
  optional path_prefix/lang), replacing the TESTING draft endpoint
- robustness: raw socket errors from a dead Solr server map to
  solr_unavailable instead of a 500
- indexing fix (to the already-merged foundations): Image content is
  excluded from chunking - its only chunkable text is title/
  description, which polluted answer sources with photos; File stays
  included (locates real documents; post-MVP Tika text builds on it)
- docs: Step 3 design decisions and known issues (shared test Solr
  core, topK source tail) in IMPLEMENTATION-79.md

Verified against the bundled German demo corpus: golden questions
19/19 retrieval hits, 2/2 declines, no binary content cited as
sources; permission trimming holds (private pages absent for the
plain member).
…cing them (#79)

With the toggle on, only @Rag-Search was fired and the classic result
list displayed a misleading 'No results'. Now both run: the answer
renders above the normal result list, matching the intended final UX.
@reekitconcept
reekitconcept merged commit 7f5ba8c into feature-ai-rag Jul 23, 2026
17 checks passed
@reekitconcept
reekitconcept deleted the kid/458/query-pipeline branch July 23, 2026 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant