Hangs during most generation requests (possibly due to disabling read and write http client timeouts?)

[ed-irl]

Environment details

• Programming language: Kotlin/JVM
• OS: MacOS
• Language runtime version: Kotlin 1.9 Java 21
• Package version: 1.17.0

Steps to reproduce

I am trying to switch to the GenAI SDK from the java SDK. I have an issue with a nondeterministic hang in a test that invokes the endpoint. I am using gemini-2.5-flash-lite. I think this might be due to the fact that read and write timeouts are unconditionally disabled in the http client here:

java-genai/src/main/java/com/google/genai/ApiClient.java

Line 224 in ec323d1

builder.readTimeout(Duration.ofMillis(0));

Regardless of whether or not this is also service error, it seems unreasonable to configure the client in such a way that it hangs indefinitely. It seems like there should be some timeout (even if it is a long one) and that it should be configurable.

I am using the global endpoint, primarily to try to overcome 429 errors that I get in us-central1.

Here are some details from the hanging requests. I've replaced the prompt sections with <text removed>:

Request content:

[Content{parts=Optional[[Part{videoMetadata=Optional.empty, thought=Optional.empty, inlineData=Optional.empty, fileData=Optional.empty, thoughtSignature=Optional.empty, functionCall=Optional.empty, codeExecutionResult=Optional.empty, executableCode=Optional.empty, functionResponse=Optional.empty, text=Optional[Create a beginner chest and back workout with dumbbells only]}]], role=Optional[user]}]

Model config:

GenerateContentConfig{httpOptions=Optional.empty, shouldReturnHttpResponse=Optional.empty, systemInstruction=Optional[Content{parts=Optional[[Part{videoMetadata=Optional.empty, thought=Optional.empty, inlineData=Optional.empty, fileData=Optional.empty, thoughtSignature=Optional.empty, functionCall=Optional.empty, codeExecutionResult=Optional.empty, executableCode=Optional.empty, functionResponse=Optional.empty, text=Optional[<text removed>]}]], role=Optional.empty}], temperature=Optional.empty, topP=Optional.empty, topK=Optional.empty, candidateCount=Optional[1], maxOutputTokens=Optional.empty, stopSequences=Optional.empty, responseLogprobs=Optional.empty, logprobs=Optional.empty, presencePenalty=Optional.empty, frequencyPenalty=Optional.empty, seed=Optional.empty, responseMimeType=Optional[text/plain], responseSchema=Optional.empty, responseJsonSchema=Optional.empty, routingConfig=Optional.empty, modelSelectionConfig=Optional.empty, safetySettings=Optional[[SafetySetting{method=Optional.empty, category=Optional[HARM_CATEGORY_DANGEROUS_CONTENT], threshold=Optional[BLOCK_ONLY_HIGH]}]], tools=Optional[[Tool{functionDeclarations=Optional[[FunctionDeclaration{behavior=Optional.empty, description=Optional[<text removed>], name=Optional[<text removed>], parameters=Optional[Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional.empty, items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional[{start_date=Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional[date-time], items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional.empty, propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[STRING]}, end_date=Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional[date-time], items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional.empty, propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[STRING]}, limit=Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional[int32], items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional.empty, propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[INTEGER]}}], propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[OBJECT]}], parametersJsonSchema=Optional.empty, response=Optional.empty, responseJsonSchema=Optional.empty}, FunctionDeclaration{behavior=Optional.empty, description=Optional[<text removed>], name=Optional[getProgramHistory], parameters=Optional[Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional.empty, items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional[{start_date=Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional[date-time], items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional.empty, propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[STRING]}, end_date=Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional[date-time], items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional.empty, propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[STRING]}, limit=Schema{anyOf=Optional.empty, default_=Optional.empty, description=Optional.empty, enum_=Optional[[]], example=Optional.empty, format=Optional[int32], items=Optional.empty, maxItems=Optional.empty, maxLength=Optional.empty, maxProperties=Optional.empty, maximum=Optional.empty, minItems=Optional.empty, minLength=Optional.empty, minProperties=Optional.empty, minimum=Optional.empty, nullable=Optional.empty, pattern=Optional.empty, properties=Optional.empty, propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[INTEGER]}}], propertyOrdering=Optional.empty, required=Optional.empty, title=Optional.empty, type=Optional[OBJECT]}], parametersJsonSchema=Optional.empty, response=Optional.empty, responseJsonSchema=Optional.empty}, FunctionDeclaration{behavior=Optional.empty, description=Optional[<text removed>], parameters=Optional.empty, parametersJsonSchema=Optional.empty, response=Optional.empty, responseJsonSchema=Optional.empty}]], retrieval=Optional.empty, googleSearch=Optional.empty, googleSearchRetrieval=Optional.empty, enterpriseWebSearch=Optional.empty, googleMaps=Optional.empty, urlContext=Optional.empty, functions=Optional.empty, computerUse=Optional.empty, codeExecution=Optional.empty}]], toolConfig=Optional[ToolConfig{functionCallingConfig=Optional[FunctionCallingConfig{mode=Optional[ANY], allowedFunctionNames=Optional.empty}], retrievalConfig=Optional.empty}], labels=Optional.empty, cachedContent=Optional.empty, responseModalities=Optional.empty, mediaResolution=Optional.empty, speechConfig=Optional.empty, audioTimestamp=Optional.empty, automaticFunctionCalling=Optional.empty, thinkingConfig=Optional.empty}

[nireekshak]

Hi @ed-irl, Thank you for bringing this to our attention. I've escalated this issue to the team, regarding the read and write timeouts and the timeout configuration. I'll share further updates with you as soon as I hear back from the team. Thanks again!

[stanley-coursera]

@nireekshak Aside from read/write, callTimeout would also be very important for ease of use and controlling the overall time of the request. Is this something we can also add as part of configuration?

[ed-irl]

FWIW, I think if you wanted to only set one timeout setting, callTimeout would be the right one (not connectTimeout).

Either fix (adding all timeouts to the config object, or switching to callTimeout from connectTimeout) is pretty simple. @nireekshak if the team had an opinion on which way they'd like to go I'd happily submit a PR. What do you think?

[ed-irl]

I've proposed two different fixes (pr #603 and #602 ) for your convenience.