aiohttp TimeoutError/460 status code

[gryevns]

I am using FastAPI via GCP Cloud Run where a aiohttp.ClientSession is created on application startup (aiohttp 3.12.15).

Infrequently I see asyncio.TimeoutError raised after 60 seconds given the following code:

ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
ssl_context.load_cert_chain(certfile=..., keyfile=...)
async with client_session.request(
    method="POST",
    url="...",
    json={...},
    headers={"content-type": "application/json; charset=utf-8"},
    ssl_context=ssl_context,
    timeout=60,
) as response:
    data = await response.json()

I have been discussing these timeouts with the third party API. From their logs, the response fulfilled in 2/3 seconds but they do see a 460 status code (AWS) which means the client connection was closed/terminated.

So far I've been unable to replicate, I've tried tweaking TLS & socket settings to no avail.

99% of the time requests work as expected but I have occasional timeouts even though the third party says all looks good their end.

Does anyone have any advice what to try next or how to debug further?

Thanks!

[Dreamsorcerer]

First, it would be good to use ClientTimeout as documented, not the deprecated int format.

But, my best guess is that because this is a total timeout, the cause may not be network related, but you may have blocking code that's stopping aiohttp from returning to the request in time. e.g. If you start the request, but have 60+ tasks queued in the event loop, which all take a full second of blocking code to run, then it won't get back to the request before the timeout period is reached.

If you've not done so already, I suggest using debug mode during development which will create warnings if any code blocks the event loop for more than 100ms:
https://docs.python.org/3/library/asyncio-dev.html

[Dreamsorcerer]

I think the only thing kind of specific to FastAPI in this regard, is if you use pydantic.EmailStr. As I understand it, this will make a blocking network request and hang your entire web server while it does a DNS lookup.

[gryevns]

Sorry, code example mistake on my end. The timeout is constructed as such aiohttp.ClientTimeout(total=60).

I quickly enabled PYTHONASYNCIODEBUG=1 but found nothing immediately obvious running the same logic, this will take some more time to debug given different services running locally. Thanks for the pointer.