Skip to content

feat: add offset pagination to find - #5337

Open
Eijnewgnaw wants to merge 1 commit into
opendatalab:nextfrom
Eijnewgnaw:codex/find-offset-pagination
Open

feat: add offset pagination to find#5337
Eijnewgnaw wants to merge 1 commit into
opendatalab:nextfrom
Eijnewgnaw:codex/find-offset-pagination

Conversation

@Eijnewgnaw

@Eijnewgnaw Eijnewgnaw commented Jul 28, 2026

Copy link
Copy Markdown

Motivation

mineru find supports limiting filename-search results but does not expose an offset, so callers cannot page through matching files consistently. This PR adds offset pagination to find and aligns its public interface with content search.

Closes #5301.

Modification

  • Add --offset to the mineru find CLI and forward it through the sync/async interfaces, HTTP client, server, and search service.
  • Apply extension and active/stale-file filtering before pagination, while preserving the total count of filtered matches.
  • Add a deterministic file_id tie-breaker to filename FTS ordering so repeated page requests remain stable.
  • Add interface, CLI, and filtering/pagination unit coverage.
  • Document the new option and its end-to-end pagination expectations.

BC-breaking (Optional)

No. The new offset argument defaults to 0, preserving existing behavior.

Use cases (Optional)

mineru find "report" --ext pdf --limit 20 --offset 20 --json

This returns the second page of PDF filename matches after filtering.

Testing

  • 8 focused CLI/interface/service pagination tests passed.
  • 889 remaining unit tests passed in the local environment (2 unrelated dependency-compatibility baseline tests deselected).
  • Ruff checks, formatting checks, and git diff --check passed.
  • A real local CLI/server flow with interleaved PDF/DOCX matches verified distinct, repeatable offset pages and the filtered total.

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.
  • Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
  • The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
  • CLA has been signed and all committers have signed the CLA in this PR.

@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels Jul 28, 2026
@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@Eijnewgnaw

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Jul 28, 2026

Copy link
Copy Markdown
Author

Hi maintainers, just a gentle follow-up when you have a chance. This PR addresses #5301 with end-to-end find --offset pagination, stable ordering, and pagination after filtering. The CLA and checks are passing. I’m happy to adjust the implementation based on your feedback. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant