Fallbacks don't trigger when a critical error is reported in streaming Responses endpoint

[arunmittal1]

Hello,

We're using LiteLLM as our AI Gateway with model groups configured with fallback options. We've noticed an issue with error handling when streaming responses from OpenAI models.

Observed behavior:
When critical errors occur (such as context window exceeded, rate limit errors, or PTU-related issues), they are returned in the first chunk after the stream has started, rather than as an HTTP error before streaming begins.

Our questions:

1. Can LiteLLM detect these errors and trigger a fallback to the next model in the group, even when these critical errors occur as the first chunk after streaming has started?
2. Is there a recommended workaround/best practice to handle this scenario, preferably without requiring clients to implement their own recovery logic?

Any guidance would be appreciated!