Unhandled Empty Response on generateContentStream due to truncated/malformed API response

[dannielcordeiro]

Context:
I am using the library in a web application built with Spring Boot 3.5.5 and Java 21. The application calls the Gemini API using the models.generateContentStream() method.

The Problem and Steps to Reproduce:
Intermittently, the application fails while consuming the response stream, sometimes just ending the stream with a empty response, sometimes throwing a GenAiIOException caused by a com.fasterxml.jackson.core.io.JsonEOFException. This occurs at the end of a stream that otherwise appears to have been received successfully, sometimes with partial responses already received, sometimes with groundingMetada or "thoughts" only, sometimes before any answer is read. I have identified that it usually happend when the API returns a 200 OK, and in the subsequent response a "error" JSON object that is not correctly treated in te "readNextJson" method of the ResponseStream class:

`---------- API RESPONSE START ----------
URL: https://generativelanguage.googleapis.com/v1alpha/models/gemini-2.5-pro:streamGenerateContent?alt=sse
2025-08-27T23:34:15.735-03:00 DEBUG 19848 --- [Http TaskRunner] okhttp3.internal.concurrent.TaskRunner : Q10005 starting : OkHttp generativelanguage.googleapis.com ping
Code: 200
Headers:
content-type: text/event-stream
content-disposition: attachment
vary: Origin
vary: X-Origin
vary: Referer
date: Thu, 28 Aug 2025 02:33:56 GMT
server: scaffolding on HTTPServer2
x-xss-protection: 0
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
server-timing: gfet4t7; dur=3130
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

2025-08-27T23:34:15.736-03:00 DEBUG 19848 --- [leapis.com ping] okhttp3.internal.http2.Http2 : >> 0x00000000 8 PING ACK
Body:
data: {"candidates": [{"content": {"parts": [{"text": "Investigating Relevant Tax Law\n\nI've initiated a deep dive into current tax law topics pertinent to high-level public examinations, with a particular focus on the Public Defender's Office's context. Initial Google searches are underway, and I'm crafting refined search queries to pinpoint the most relevant and recent developments. I'm aiming for actionable insights to inform the next steps of my analysis.\n\n\n","thought": true}],"role": "model"},"index": 0,"groundingMetadata": {}}],"usageMetadata": {"promptTokenCount": 1358,"totalTokenCount": 1404,"promptTokensDetails": [{"modality": "TEXT","tokenCount": 1358}],"thoughtsTokenCount": 46},"modelVersion": "gemini-2.5-pro","responseId": "FMCvaO2mCPysz7IP4PzPOA"}

...

depth analysis.\n\n\n","thought": true}],"role": "model"},"index": 0,"groundingMetadata": {}}],"usageMetadata": {"promptTokenCount": 1358,"totalTokenCount": 1604,"promptTokensDetails": [{"modality": "TEXT","tokenCount": 1358}],"thoughtsTokenCount": 246},"modelVersion": "gemini-2.5-pro","responseId": "FMCvaO2mCPysz7IP4PzPOA"}

{
"error": {
"code": 503,
"message": "The model is overloaded. Please try again later.",
"status": "UNAVAILABLE"
}
}

`

Above is a snippet of a interceptor log I placed on the ApiClient class to debug the issue.

I also believe that this behavior is causing the connection to the Gemini API to not be properly closed, since after the first error I commonly get in the application logs:

2025-08-27T09:13:08.076-03:00 DEBUG 36988 --- [.googleapis.com] okhttp3.internal.http2.Http2 : << 0x00000000 25 GOAWAY

and

connection to https://generativelanguage.googleapis.com/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE

I believe that the GenIAException is not causing the connection to close, as it should (or maybe throw an ServerException 503 in this scenario), and not closing the stream either. It seems that, when in a "try ()" clause, the GenAiIOException is not caught in the Exception clause and the streams just finishes "normally", without any actual responses in most cases.

Environment details

• Programming language: Java
• OS: Windows (11) and Ubuntu 24
• Language runtime version: 21
• Package version: 1.15.0