feat: optional query log for LeannSearcher (#70)#325
Open
raoabinav wants to merge 2 commits into
Open
Conversation
Closes yichuan-w#70. LeannSearcher reads LEANN_QUERY_LOG from the environment on init. When set, each call to search() appends a JSON line to that file with the query text, the query embedding (when one was computed — pure-BM25 paths skip it), top_k, and the result id/score pairs. Failures appending are logged and don't break search. The format is jsonl, append-only, no rotation. Off by default — has to be explicitly opted in via the env var, so existing callers aren't affected. I went with an env var rather than a CLI flag because the issue framed this as a researcher use case and env-var fits "set it once for the session, forget about it". Happy to add a --log-queries CLI flag too if that's preferred. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's ty caught that `open(self._query_log_path, ...)` is invalid because `self._query_log_path` is typed `Optional[str]` and `open()` doesn't accept None. The runtime guard `if self._query_log_path:` at the call site doesn't narrow inside the called method. Local variable narrows the type cleanly. Runtime behavior unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #70.
LeannSearcherreadsLEANN_QUERY_LOGfrom the environment on init. When set, each call tosearch()appends a JSON line to that file with the query text, the query embedding (when one was computed, pure-BM25 paths skip it),top_k, and the result id/score pairs. Failures appending get logged at WARNING and don't break the search.