Skip to content

fix: support filename prefix search - #5366

Open
lylelllll wants to merge 1 commit into
opendatalab:nextfrom
lylelllll:codex/fix-filename-prefix-search
Open

fix: support filename prefix search#5366
lylelllll wants to merge 1 commit into
opendatalab:nextfrom
lylelllll:codex/fix-filename-prefix-search

Conversation

@lylelllll

@lylelllll lylelllll commented Aug 6, 2026

Copy link
Copy Markdown

Motivation

mineru find bench currently misses filenames such as bench1.pdf, even though the explicit wildcard form bench* works. This makes the default filename search behavior surprising and is the issue reported in #5281.

Modification

  • Apply FTS5 prefix matching to the final sanitized token in filename searches when the query does not already end in *.
  • Preserve explicit wildcard queries and keep content full-text search behavior unchanged.
  • Add SQLite-backed regression coverage for implicit and explicit prefixes, exact tokens, multi-token queries, CJK filenames, and filename/content search isolation.

BC-breaking (Optional)

No. Filename lookup becomes more permissive for the final query token; public Python APIs, content search, indexing, and storage are unchanged.

Testing

  • .venv/bin/python -m pytest -q -o addopts='' tests/unittest/test_doclib_cache_semantics.py tests/unittest/test_cli_next_command_design.py (307 passed)
  • uvx ruff check mineru/doclib/core/fts.py
  • uvx ruff check --ignore ANN001,E501 tests/unittest/test_doclib_cache_semantics.py
  • Direct tokenization smoke test for implicit prefix, explicit wildcard, multi-token, and CJK queries

Model/GPU parsing E2E tests were not run because this change is isolated to SQLite FTS filename query construction.

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • Documentation impact was reviewed; no documentation change is needed for this targeted bug fix.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects. (No downstream API or storage change.)
  • CLA has been signed and all committers have signed the CLA in this PR.

Fixes #5281

@dosubot dosubot Bot added size:XS This PR changes 0-9 lines, ignoring generated files. bug Something isn't working labels Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:XS This PR changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant