RAG query pipeline: @rag-search retrieval endpoint + answer generation (#79) - #90
Merged
Merged
Conversation
#79) Step 3 of the RAG MVP (#458): a question to @Rag-Search returns {answer, sources} end to end. - 3.1 grounded prompt template: answer strictly from the provided context, decline when the answer is not there, reply in the question's language; <think> blocks stripped from the output - 3.2 rag/pipeline.py: run_rag_search - embed the question, {!knn} chunk retrieval with the same security/path/language pre-filters as the classic search, collapse chunks to parent documents, generate; RagResult with error codes (not_configured/embedding_failed/ generation_failed/solr_unavailable); generation failure still exposes the retrieved sources; empty retrieval is not an error - 3.3 thin @Rag-Search restapi service on the pipeline (q required, optional path_prefix/lang), replacing the TESTING draft endpoint - robustness: raw socket errors from a dead Solr server map to solr_unavailable instead of a 500 - indexing fix (to the already-merged foundations): Image content is excluded from chunking - its only chunkable text is title/ description, which polluted answer sources with photos; File stays included (locates real documents; post-MVP Tika text builds on it) - docs: Step 3 design decisions and known issues (shared test Solr core, topK source tail) in IMPLEMENTATION-79.md Verified against the bundled German demo corpus: golden questions 19/19 retrieval hits, 2/2 declines, no binary content cited as sources; permission trimming holds (private pages absent for the plain member).
…cing them (#79) With the toggle on, only @Rag-Search was fired and the classic result list displayed a misleading 'No results'. Now both run: the answer renders above the normal result list, matching the intended final UX.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step 3 of the RAG MVP (ticket: distribution-kitconcept-intranet#458, part of #79): a question to
@rag-searchreturns{answer, sources}end to end. Second commit is the removableRAG: TESTING:UI aid.What's included
<think>blocks strippedrag/pipeline.py): embed the question →{!knn}chunk retrieval with the same security/path/language pre-filters as the classic search → collapse chunks to parent documents → generate.RagResultwith error codes (not_configured/embedding_failed/generation_failed/solr_unavailable); generation failure still exposes the retrieved sources; empty retrieval is not an error@rag-searchrestapi service on the pipeline (qrequired → 400, optionalpath_prefix/lang), replacing the TESTING draft endpointsolr_unavailableinstead of a 500Imagecontent is excluded from chunking — its only chunkable text is title/description, which polluted answer sources with photos;Filestays included (its title/description locates real documents, and the post-MVP Tika body extraction builds on it)Verification
Against the bundled German demo corpus (#89): golden questions 19/19 retrieval hits, 2/2 declines, no binary content cited as sources after the indexing fix; permission trimming holds (private pages absent for the plain member
f.meier); RAG test suite 110 passed.