fix(test): fix AWS_IO_DNS_INVALID_NAME in multiprocess tests

[jet-tong]

Description

• Add multiprocessing_context=mp.get_context() to force DataLoader to use the same context as the main process (i.e. spawn instead of fork on Darwin) for multiprocessing e2e tests,
• Prevents S3 client fork handlers from corrupting AWS CRT DNS resolver threads on macOS GitHub runners.

Alternatively, we can also consider changing behaviour on S3Client's fork handlers specifically on Darwin platform.

Additional context

Error Context

Tests were failing with AWS_IO_DNS_INVALID_NAME errors on macos-14 (ARM64) GitHub Actions runners during our Build Wheels workflow, particularly in test_distributed_training.py. These errors were intermittent, across 5 macos-14 runners each running 40+ tests, typically 1-2 fails. We could resolve these failures by retrying the Build Wheels workflow 1-2 times for every run.

The error re-appeared in PR #374. The error stack traces showed popen_fork.py being used, which was suspicious since we were explicitly setting torch.multiprocessing.set_start_method('spawn'), and disabled fork / forkserver methods for Darwin.

Root Cause

PyTorch DataLoader source code (torch/utils/data/dataloader.py) showed that when multiprocessing_context=None (the default), DataLoader ignores the global torch.multiprocessing.set_start_method() setting and uses its own default context. This means DataLoader was using fork internally, triggering the S3 client's fork handlers.

When DataLoader forks workers, S3 client fork handlers (registered via os.register_at_fork() as a workaround for CRT multithreading issues in PR #320) call join_all_managed_threads() to clean up AWS CRT background threads, but this corrupts active DNS resolver threads and causes AWS_IO_DNS_INVALID_NAME errors.

• I have updated the CHANGELOG or README if appropriate

Related items

• Related PR with errors: ci: migrate from macos-13 to macos-15-intel image #374
• PR for workaround for fork: New work around for fork with multithread from CRT #320

Testing

Build Wheels workflow: https://github.com/awslabs/s3-connector-for-pytorch/actions/runs/18225016406

---

By submitting this pull request, I confirm that my contribution is made under the terms of BSD 3-Clause License and I agree to the terms of the LICENSE.

[jet-tong]

test_logging.py subprocess.Popen could also cause the error.

We can also consider changing behaviour on S3Client's fork handlers specifically on Darwin platform.