Description
After upgrading from s3fs 2025.10.0 to 2026.2.0, custom before-sign.s3 event handlers registered on fs.s3.meta.events are silently lost when performing S3 operations in synchronous mode.
Steps to reproduce
from botocore import UNSIGNED
from s3fs import S3FileSystem
class TestSigner:
def __init__(self, properties):
print("[INIT]")
def __call__(self, request, **kwargs):
print(f"[CALL] {request.method} {request.url}")
fs = S3FileSystem(
anon=False,
endpoint_url="https://example-bucket.s3.us-east-1.amazonaws.com",
client_kwargs={"region_name": "us-east-1"},
config_kwargs={"signature_version": UNSIGNED},
)
signer = TestSigner({})
fs.s3.meta.events.unregister("before-sign.s3", unique_id=1925)
fs.s3.meta.events.register_last("before-sign.s3", signer, unique_id=1925)
print(f"Emitter id before operation: {id(fs.s3.meta.events._emitter)}")
try:
fs.ls("s3://")
except Exception:
pass
print(f"Emitter id after operation: {id(fs.s3.meta.events._emitter)}")
s3fs 2025.10.0 output (expected):
[INIT]
Emitter id before operation: 4480874448
[CALL] GET https://example-bucket.s3.us-east-1.amazonaws.com/
Emitter id after operation: 4480874448 # same emitter, handler fired
s3fs 2026.2.0 output (bug):
[INIT]
Emitter id before operation: 4456644096
Emitter id after operation: 4543285056 # <-- different emitter, [CALL] NEVER printed
Root cause
|
if self._s3 is not None and not refresh: |
|
hsess = getattr(getattr(self._s3, "_endpoint", None), "http_session", None) |
|
if hsess is not None: |
|
if hsess._sessions is None or ( |
|
hsess._sessions and all(_.closed for _ in hsess._sessions.values()) |
|
): |
|
refresh = True |
|
if not refresh: |
|
return self._s3 |
set_session() now checks whether aiohttp sessions are closed, and forces refresh=True if so:
In synchronous mode (asynchronous=False), connect() runs set_session() inside a temporary event loop via sync_wrapper. When that loop exits, aiohttp sessions are closed. On the next S3 operation, _call_s3 → set_session() detects the closed sessions → refresh=True → creates a brand-new S3 client via create_client() → all event handlers registered on the old client's meta.events are discarded.
This breaks any downstream code that registers event handlers on fs.s3.meta.events after S3FileSystem initialization, such as custom S3 signers used for remote signing.
Environment
- s3fs: 2026.2.0
- aiobotocore: 3.6.0
- Python: 3.12
Versions affected
- 2026.2.0: bug present (session liveness check added)
- 2025.10.0: works correctly
Related issues
Description
After upgrading from s3fs 2025.10.0 to 2026.2.0, custom
before-sign.s3event handlers registered onfs.s3.meta.eventsare silently lost when performing S3 operations in synchronous mode.Steps to reproduce
s3fs 2025.10.0 output (expected):
s3fs 2026.2.0 output (bug):
Root cause
s3fs/s3fs/core.py
Lines 603 to 611 in 162f23e
set_session()now checks whether aiohttp sessions are closed, and forcesrefresh=Trueif so:In synchronous mode (
asynchronous=False),connect()runsset_session()inside a temporary event loop viasync_wrapper. When that loop exits, aiohttp sessions are closed. On the next S3 operation,_call_s3→set_session()detects the closed sessions →refresh=True→ creates a brand-new S3 client viacreate_client()→ all event handlers registered on the old client'smeta.eventsare discarded.This breaks any downstream code that registers event handlers on
fs.s3.meta.eventsafterS3FileSysteminitialization, such as custom S3 signers used for remote signing.Environment
Versions affected
Related issues