Releases: awslabs/s3-connector-for-pytorch
Releases · awslabs/s3-connector-for-pytorch
v1.4.3
v1.4.2
v1.4.2 (July 14, 2025)
New features
- Implement range-based S3 Reader for efficient partial read patterns (#339)
- Implemented AWS profile credentials support as part of S3ClientConfig (#341)
Bug fixes
- Enable multiple CRT clients per process with different configs (#340)
Other changes
- Expose configuration needed for testing profile credentials (#344)
- Keep usage of manylinux_2014 for wider suport of glibc (#342, #343)
Breaking changes
- No breaking changes.
Full Changelog: v1.4.1...v1.4.2
v1.4.1
v1.4.1 (May 20, 2025)
New features
- Add S3 prefix strategies to prevent throttling in distributed checkpointing (#327)
Bug fixes
- Consume mountpoint-s3-client 0.14.1 to address AWS_ERROR_HTTP_RESPONSE_FIRST_BYTE_TIMEOUT error (#332)
- Prevent S3Writer stream closure on exception to preserve original error context (#331)
- Add PyTorch 2.7.0 support (#329)
Other changes
- Add Python version to user agent string (#333)
- Address DCP test hanging and distributed test errors (#330)
- Update GitHub Actions runners to ubuntu-24.04 (#328)
Breaking changes
- No breaking changes.
Full Changelog: v1.4.0...v1.4.1
v1.4.0 (April 14, 2025)
v1.4.0 (April 14, 2025)
New features
- Introducing support of FSDP in DCP benchmark (#313)
- Exposing max_attempts setting through S3ClientConfig and adding support of S3ClientConfig to DCP (#322)
Bug fixes
- Consume mountpoint-s3-client changes for race condition on GET request path, that may lead to an empty response (#324)
Other changes
- Update Pyo3 version (#314)
Breaking changes
- We changed the way we handle fork operations. The CRT client used for communication with S3 is not stable during fork due to global state and locks held by background threads. To address this, we now clean up all existing CRT clients before a fork operation and create new CRT clients in the child process. This change prevents segfaults and hanging GET requests for training workloads that rely on fork (#320)
Full Changelog: v1.3.2...v1.4.0
v1.3.2
v1.3.2 (February 5, 2025)
New features
- Consume mountpoint-s3-client changes with support for dots in bucket name for COPY operation introduced in CRT (#300)
- Escape special characters in rename operation (#297)
- Support dots in bucket name for rename operation (#300)
- Handle torch.load changes in PyTorch 2.6 (#306)
- Remove dependency on mountpoint-s3-crt (#307)
Breaking changes
- Internal S3Client now returns
HeadObjectResultinstead ofObjectInfo. The only difference is thatHeadObjectResultdoesn't havekeyfield.
Full Changelog: v1.3.1...v1.3.2
v1.3.1
v1.3.1 (January 10, 2025)
New features
- Fix Rust build (#275)
- Update user agent for DCP (#276)
- Address Rust security issue (#279)
- Refactor(benchmarks): Overhaul Lightning Checkpointing, DCP, dataset scenarios; add DynamoDB writes and results exploitation notebook (#274, #280, #285, #286)
- Add single rank PyTorch checkpoint benchmark (#289)
- Update torch version restriction (<2.5.0) and bind torchdata to last version with DataPipes (#283)
Breaking changes
- No breaking changes.
Full Changelog: v1.3.0...v1.3.1
v1.3.0
v1.2.7
v1.2.7 (October 29, 2024)
New features
- Add support for CRT retries (awslabs/mountpoint-s3#1069).
- Add support for
CopyObjectAPI (#242).
v1.2.6
v1.2.5
New features
- Enhanced error logging.
- Support tell for S3writer.
- Path-style addressing support.
- Update crates and Mountpoint dependencies.
Breaking changes
- No breaking changes.
Full Changelog: v1.2.4...v1.2.5