Skip to content

recreates S3 client after sync event loop exits, discarding custom event handlers #1030

Description

@fcfangcc

Description

After upgrading from s3fs 2025.10.0 to 2026.2.0, custom before-sign.s3 event handlers registered on fs.s3.meta.events are silently lost when performing S3 operations in synchronous mode.

Steps to reproduce

from botocore import UNSIGNED
from s3fs import S3FileSystem

class TestSigner:
    def __init__(self, properties):
        print("[INIT]")

    def __call__(self, request, **kwargs):
        print(f"[CALL] {request.method} {request.url}")

fs = S3FileSystem(
    anon=False,
    endpoint_url="https://example-bucket.s3.us-east-1.amazonaws.com",
    client_kwargs={"region_name": "us-east-1"},
    config_kwargs={"signature_version": UNSIGNED},
)

signer = TestSigner({})
fs.s3.meta.events.unregister("before-sign.s3", unique_id=1925)
fs.s3.meta.events.register_last("before-sign.s3", signer, unique_id=1925)

print(f"Emitter id before operation: {id(fs.s3.meta.events._emitter)}")

try:
    fs.ls("s3://")
except Exception:
    pass

print(f"Emitter id after operation:  {id(fs.s3.meta.events._emitter)}")

s3fs 2025.10.0 output (expected):

[INIT]
Emitter id before operation: 4480874448
[CALL] GET https://example-bucket.s3.us-east-1.amazonaws.com/
Emitter id after operation:  4480874448   # same emitter, handler fired

s3fs 2026.2.0 output (bug):

[INIT]
Emitter id before operation: 4456644096
Emitter id after operation:  4543285056   # <-- different emitter, [CALL] NEVER printed

Root cause

s3fs/s3fs/core.py

Lines 603 to 611 in 162f23e

if self._s3 is not None and not refresh:
hsess = getattr(getattr(self._s3, "_endpoint", None), "http_session", None)
if hsess is not None:
if hsess._sessions is None or (
hsess._sessions and all(_.closed for _ in hsess._sessions.values())
):
refresh = True
if not refresh:
return self._s3

set_session() now checks whether aiohttp sessions are closed, and forces refresh=True if so:

In synchronous mode (asynchronous=False), connect() runs set_session() inside a temporary event loop via sync_wrapper. When that loop exits, aiohttp sessions are closed. On the next S3 operation, _call_s3set_session() detects the closed sessions → refresh=True → creates a brand-new S3 client via create_client() → all event handlers registered on the old client's meta.events are discarded.

This breaks any downstream code that registers event handlers on fs.s3.meta.events after S3FileSystem initialization, such as custom S3 signers used for remote signing.

Environment

  • s3fs: 2026.2.0
  • aiobotocore: 3.6.0
  • Python: 3.12

Versions affected

  • 2026.2.0: bug present (session liveness check added)
  • 2025.10.0: works correctly

Related issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions