Skip to content

fix(buildsilver): ValueError when first_n_pages > max_sample_pages - #466

Open
devteamaegis wants to merge 1 commit into
allenai:mainfrom
devteamaegis:fix/sample-pdf-pages-negative-k
Open

fix(buildsilver): ValueError when first_n_pages > max_sample_pages#466
devteamaegis wants to merge 1 commit into
allenai:mainfrom
devteamaegis:fix/sample-pdf-pages-negative-k

Conversation

@devteamaegis

Copy link
Copy Markdown

What's broken

sample_pdf_pages() crashes with ValueError: Sample larger than population or is negative when first_n_pages > max_sample_pages and the PDF has more pages than first_n_pages. The additional-pages count max_sample_pages - first_n_pages goes negative and is passed to random.sample(), which rejects negative sizes. --first_n_pages and --max_sample_pages are independent CLI flags, so e.g. --first_n_pages 20 with the default --max_sample_pages 15 makes every multi-page PDF crash in process_pdf.

Why it happens

min(max_sample_pages - first_n_pages, len(remaining_pages)) can be negative; nothing floors it at 0. The code assumes max_sample_pages >= first_n_pages.

Fix

Clamp the sample count with max(0, ...). The first first_n_pages are still returned; no extra sampling once the budget is exhausted.

Test

Adds tests/test_buildsilver.py covering the negative-count case (fails before), plus normal sampling and fewer-pages-than-first_n. Passes after.

When first_n_pages exceeds max_sample_pages and a PDF has more pages than
first_n_pages, the additional-pages count (max_sample_pages - first_n_pages)
is negative. random.sample() rejects a negative sample size and raises
ValueError, crashing page sampling. Clamp the count to be non-negative.

Adds tests covering the negative-count case and normal sampling.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant