v5 release roadmap (testers wanted!)

[mara004]

Our development branch for the next major release (v5) is finally nearing a state at which it could be merged into main some time soon. Highlights include:

• PdfPosConv / PdfBitmap.get_posconv(page) to translate between page and bitmap coordinates
• PdfObject.get_quad_points() to get the corner positions of an image or text object
• PdfImage.get_bitmap(render=True): scale_to_original option to preserve the native pixel size and avoid resolution degradation
• Method-oriented PdfBookmark and PdfDest helpers replace the old PdfOutlineItem namedtuple
• New rendering CLI options --invert-lightness --exclude-images to post-process with selective lightness inversion (this may be useful to achieve a "dark look" for light PDFs)
• Experimental Android (and iOS) support
• Removal of deprecated APIs, cleanups

See docs/devel/changelog_staging.md for more.

---

The branch has been around for a rather long time now, and effectively away from end-user testing. While there are many improvements, it is also possible something may have broken/regressed.
That's why we'd like downstreams to try out the updated codebase and give feedback (taking into account the intentional API-breaking changes – see the changelog).

The plan is to make a beta release (or, if necessary, several beta releases) and wait for community response for at least 2 weeks (maybe more) before committing the actual release.

For the first few follow-up releases of the v5 series (e.g. 5.1 and 5.2), we reserve the right to apply API(-changing) corrections, should the need arise. After this, we endeavor to keep the helpers API stable as we did for v4, until the next major release.

Thanks for your attention.

[mara004]

If there are no objections, I plan to merge devel_new into main tomorrow.

[mara004]

All right, I've merged the devel branch, and the first beta release has rolled out:
https://github.com/pypdfium2-team/pypdfium2/releases/tag/5.0.0b1
https://pypi.org/project/pypdfium2/5.0.0b1/

To install, you can do e.g.
python3 -m pip install --pre pypdfium2 or maybe
python3 -m pip install "git+https://github.com/pypdfium2-team/pypdfium2@main"

Please test – thanks!

[mara004]

On #307 (comment)

Once it's stabilized i would create a pre-release or release candidate and start receiving feedback from there, there's no future without breaking changes ;)

@semoal @saveliyvasilev That day has finally come – we've made the pre-release as you suggested. Testers welcome!

[mara004]

Over two weeks have passed, but there has not been any feedback yet. It seems like beta releases do not get much community attention.
Maybe we just need to go ahead with the real release and take feedback from that onward. If something is not right, we can still fix it in a follow-up release as usual.

[yqliving]

I am using the PDF flatten function in the beta release (5.0.0b1), it works well for me. May I know when the stable version could be available? Thanks.

[mara004]

Thanks for the info. There is no set timeframe. I will release the first stable (non-beta) version once I feel like the code is ready.

I plan to merge #347 soon and (if no new problems come up in the meantime) make a release once I have confirmation from the reporter if it actually fixes #346. Whether this will already be stable v5.0.0 or maybe another beta release (5.0.0b2) I'm not sure yet. Tentatively, I think it may be v5.0.0 as we have some positive feedback now.

[Goldziher]

Saw this now.

I can add this to my test matrix for kreuzberg

[Goldziher]

I can confirm this release works cleanly on my end.

[mara004]

Thanks for the feedback!

[yqliving]

I found one issue with the page flatten() function. Text extraction after flattening does not keep the correct reading order. The text from the fillable area is listed after all the general text, but it should keep the original order.
related functions:

        page.flatten()
        page = pdf[0]
        textpage = page.get_textpage()
        text_all = textpage.get_text_range()    

[mara004]

Please file a bug report for that, including the PDF in question and the output.
pypdfium2 is only forwarding the text buffer from pdfium, so I suppose you can submit the issue to upstream directly.

[mara004]

If there are no objections, I plan to make the 5.0.0 release soon.

[mara004]

Apologies for the delay, I got immersed in the native sourcebuild (#350, #352) and system pdfium fallback (#353), which should further improve portability. I'd like to wait with v5 until this work is finished.

Also, new releases are currently blocked on #354.

[mara004]

Another status update: The above PRs and issues have been addressed (+ SLSA integration and ctypesgen improvements), but I'm now brooding on #328, which would be nice to land in the initial v5 release as well.

Also, the Readme needs revising, see #361 for a stub.