Skip to content

Python 3.14: PEP-784 compression.zstd #14129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

Rogdham
Copy link
Contributor

@Rogdham Rogdham commented May 24, 2025

Second PR about PEP-784 in stdlib (first PR):

  • Add type hints for _zstd
  • Add type hints for compression.zstd

The CI current failures are due to the use of 3.14 beta1; to address this:

  • Make sure tests failing in CI pass with local build of CPython 3.14 (3.14 branch)
  • Wait for CPython 3.14 beta2 to be released (this should fix the stubtest errors) - expected May 26th
  • Re-run CI to make sure everything is ok after beta2

This comment has been minimized.

This comment has been minimized.

@Rogdham
Copy link
Contributor Author

Rogdham commented May 24, 2025

@emmatyping: if you want to give try this out, or simply comment on the use of Mapping[CompressionParameter, int] and Mapping[DecompressionParameter, int] as type hints for the options parameter. Any feedback would be appreciated!

@emmatyping
Copy link
Member

Note that I typed the options as Mapping[CompressionParameter, int] / Mapping[DecompressionParameter, int], while a Mapping[int, int] is technically accepted.

I think we should type that it accepts Mapping[int, int] as well. I consider that a supported usage.

stdlib/_zstd.pyi Outdated
def __init__(
self, level: int | None = ..., options: _OptionsCompress | None = ..., zstd_dict: ZstdDict | None = ...
) -> None: ...
def compress(self, /, data: ReadableBuffer, mode: Literal[0, 1, 2] = ...) -> bytes: ...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be useful to express these in terms of the ZstdCompressor attributes. It will make it easier to understand I think? If you think it makes the signature too busy then I suppose it is fine as is, but I want discourage passing bare integers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(similarly it'd be good to do the same below with the ZstdFile methods)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the below is invalid:

Literal[CONTINUE, FLUSH_BLOCK, FLUSH_FRAME]

But maybe I can try to do something like the following:

_ZstdCompressorContinueType = Literal[0]
_ZstdCompressorFlushBlockType = Literal[1]
_ZstdCompressorFlushFrameType = Literal[2]

@final
class ZstdCompressor:
    CONTINUE: Final[_ZstdCompressorContinueType] = 0
    FLUSH_BLOCK: Final[_ZstdCompressorFlushBlockType] = 1
    FLUSH_FRAME: Final[_ZstdCompressorFlushFrameType] = 2
    def compress(self, /, data: ReadableBuffer, mode: _ZstdCompressorContinueType | _ZstdCompressorFlushBlockType | _ZstdCompressorFlushFrameType = ...) -> bytes: ...

Is it what you meant?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't you do CONTINUE | FLUSH_BLOCK | FLUSH_FRAME?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's possible (playground)

main.py:8: error: Variable "__main__.ZstdCompressor.CONTINUE" is not valid as a type  [valid-type]
main.py:8: note: See https://mypy.readthedocs.io/en/stable/common_issues.html#variables-vs-type-aliases
main.py:8: error: Variable "__main__.ZstdCompressor.FLUSH_BLOCK" is not valid as a type  [valid-type]
main.py:8: error: Variable "__main__.ZstdCompressor.FLUSH_FRAME" is not valid as a type  [valid-type]

I will go ahead and make the changes of defining the Literal once and reusing them afterwards.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I think that is fine then.

Copy link
Contributor

According to mypy_primer, this change has no effect on the checked open source code. 🤖🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants