Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 54 additions & 5 deletions docs/streaming/custom-streaming-ws.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ In order to use voice/video streaming in ADK, you will need to use Gemini models

There is also a [SSE](custom-streaming.md) version of the sample is available.

## 1. Install ADK { #install-adk }
## 1. Install ADK {#1.-setup-installation}

Create & Activate Virtual Environment (Recommended):

Expand Down Expand Up @@ -64,7 +64,7 @@ adk-streaming-ws/
└── agent.py # Agent definition
```

## 2\. Set up the platform { #set-up-the-platform }
## 2\. Set up the platform {#2.-set-up-the-platform}

To run the sample app, choose a platform from either Google AI Studio or Google Cloud Vertex AI:

Expand Down Expand Up @@ -125,7 +125,7 @@ Notice how easily you integrated [grounding with Google Search](https://ai.googl

![intro_components.png](../assets/quickstart-streaming-tool.png)

## 3\. Interact with Your Streaming app { #interact-with-your-streaming-app }
## 3\. Interact with Your Streaming app {#3.-interact-with-your-streaming-app}

1\. **Navigate to the Correct Directory:**

Expand Down Expand Up @@ -180,7 +180,7 @@ These console logs are important in case you develop your own streaming applicat
- **When `ws://` doesn't work:** If you see any errors on the Chrome DevTools with regard to `ws://` connection, try replacing `ws://` with `wss://` on `app/static/js/app.js` at line 28. This may happen when you are running the sample on a cloud environment and using a proxy connection to connect from your browser.
- **When `gemini-2.0-flash-exp` model doesn't work:** If you see any errors on the app server console with regard to `gemini-2.0-flash-exp` model availability, try replacing it with `gemini-2.0-flash-live-001` on `app/google_search_agent/agent.py` at line 6.

## 4. Server code overview { #server-code-overview }
## 4. Server code overview {#4.-server-side-code-overview}

This server app enables real-time, streaming interaction with ADK agent via WebSockets. Clients send text/audio to the ADK agent and receive streamed text/audio responses.

Expand Down Expand Up @@ -245,6 +245,12 @@ async def start_agent_session(user_id, is_audio=False):
# Set response modality
modality = "AUDIO" if is_audio else "TEXT"
run_config = RunConfig(response_modalities=[modality])

# Optional: Enable session resumption for improved reliability
# run_config = RunConfig(
# response_modalities=[modality],
# session_resumption=types.SessionResumptionConfig()
# )

# Create a LiveRequestQueue for this session
live_request_queue = LiveRequestQueue()
Expand Down Expand Up @@ -276,6 +282,49 @@ This function initializes an ADK agent live session.

**Returns:** `(live_events, live_request_queue)`.

### Session Resumption Configuration

ADK supports live session resumption to improve reliability during streaming conversations. This feature enables automatic reconnection when live connections are interrupted due to network issues.

#### Enabling Session Resumption

To enable session resumption, you need to:

1. **Import the required types**:
```py
from google.genai import types
```

2. **Configure session resumption in RunConfig**:
```py
run_config = RunConfig(
response_modalities=[modality],
session_resumption=types.SessionResumptionConfig()
)
```

#### Session Resumption Features

- **Automatic Handle Caching** - The system automatically caches session resumption handles during live conversations
- **Transparent Reconnection** - When connections are interrupted, the system attempts to resume using cached handles
- **Context Preservation** - Conversation context and state are maintained across reconnections
- **Network Resilience** - Provides better user experience during unstable network conditions

#### Implementation Notes

- Session resumption handles are managed internally by the ADK framework
- No additional client-side code changes are required
- The feature is particularly beneficial for long-running streaming conversations
- Connection interruptions become less disruptive to the user experience

#### Troubleshooting

If you encounter errors with session resumption:

1. **Check model compatibility** - Ensure you're using a model that supports session resumption
2. **API limitations** - Some session resumption features may not be available in all API versions
3. **Remove session resumption** - If issues persist, you can disable session resumption by removing the `session_resumption` parameter from `RunConfig`

### `agent_to_client_messaging(websocket, live_events)`

```py
Expand Down Expand Up @@ -440,7 +489,7 @@ async def websocket_endpoint(websocket: WebSocket, user_id: int, is_audio: str):
* `agent_to_client_messaging`: ADK `live_events` -> Client WebSocket.
4. Bidirectional streaming continues until disconnection or error.

## 5. Client code overview { #client-code-overview }
## 5. Client code overview {#5.-client-side-code-overview}

The JavaScript `app.js` (in `app/static/js`) manages client-side interaction with the ADK Streaming WebSocket backend. It handles sending text/audio and receiving/displaying streamed responses.

Expand Down
66 changes: 60 additions & 6 deletions docs/streaming/custom-streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This article overviews the server and client code for a custom asynchronous web

There is also a [WebSocket](custom-streaming-ws.md) version of the sample is available.

## 1. Install ADK { #install-adk }
## 1. Install ADK {#1.-setup-installation}

Create & Activate Virtual Environment (Recommended):

Expand Down Expand Up @@ -69,7 +69,7 @@ adk-streaming/
└── agent.py # Agent definition
```

## 2\. Set up the platform { #set-up-the-platform }
## 2\. Set up the platform {#2.-set-up-the-platform}

To run the sample app, choose a platform from either Google AI Studio or Google Cloud Vertex AI:

Expand Down Expand Up @@ -105,7 +105,7 @@ To run the sample app, choose a platform from either Google AI Studio or Google
```


## 3\. Interact with Your Streaming app { #interact-with-your-streaming-app }
## 3\. Interact with Your Streaming app {#3.-interact-with-your-streaming-app}

1\. **Navigate to the Correct Directory:**

Expand Down Expand Up @@ -158,7 +158,7 @@ These console logs are important in case you develop your own streaming applicat
- **When your browser can't connect to the server via SSH proxy:** SSH proxy used in various cloud services may not work with SSE. Please try without SSH proxy, such as using a local laptop, or try the [WebSocket](custom-streaming-ws.md) version.
- **When `gemini-2.0-flash-exp` model doesn't work:** If you see any errors on the app server console with regard to `gemini-2.0-flash-exp` model availability, try replacing it with `gemini-2.0-flash-live-001` on `app/google_search_agent/agent.py` at line 6.

## 4. Agent definition { #agent-definition }
## 4. Agent definition

The agent definition code `agent.py` in the `google_search_agent` folder is where the agent's logic is written:

Expand All @@ -184,7 +184,7 @@ Notice how easily you integrated [grounding with Google Search](https://ai.googl

The server and client architecture enables real-time, bidirectional communication between web clients and AI agents with proper session isolation and resource management.

## 5. Server side code overview { #server-side-code-overview }
## 5. Server side code overview {#5.-server-side-code-overview}

The FastAPI server provides real-time communication between web clients and the AI agent.

Expand All @@ -210,6 +210,11 @@ The FastAPI server provides real-time communication between web clients and the
- **Stream Resilience** - SSE streams handle exceptions and perform cleanup automatically
- **Connection Recovery** - Clients can reconnect by re-establishing SSE connection

#### Session Resumption:
- **Live Session Resumption** - Enables transparent reconnection to interrupted live conversations
- **Handle Caching** - System automatically caches session handles for recovery
- **Reliability Enhancement** - Improves resilience against network instability during streaming


### Agent Session Management

Expand All @@ -234,6 +239,12 @@ async def start_agent_session(user_id, is_audio=False):
# Set response modality
modality = "AUDIO" if is_audio else "TEXT"
run_config = RunConfig(response_modalities=[modality])

# Optional: Enable session resumption for improved reliability
# run_config = RunConfig(
# response_modalities=[modality],
# session_resumption=types.SessionResumptionConfig()
# )

# Create a LiveRequestQueue for this session
live_request_queue = LiveRequestQueue()
Expand Down Expand Up @@ -382,6 +393,49 @@ async def sse_endpoint(user_id: int, is_audio: str = "false"):

- **Cleanup Logic** - Handles connection termination by closing the request queue and removing from active sessions, with error handling for stream interruptions.

### Session Resumption Configuration

ADK supports live session resumption to improve reliability during streaming conversations. This feature enables automatic reconnection when live connections are interrupted due to network issues.

#### Enabling Session Resumption

To enable session resumption, you need to:

1. **Import the required types**:
```py
from google.genai import types
```

2. **Configure session resumption in RunConfig**:
```py
run_config = RunConfig(
response_modalities=[modality],
session_resumption=types.SessionResumptionConfig()
)
```

#### Session Resumption Features

- **Automatic Handle Caching** - The system automatically caches session resumption handles during live conversations
- **Transparent Reconnection** - When connections are interrupted, the system attempts to resume using cached handles
- **Context Preservation** - Conversation context and state are maintained across reconnections
- **Network Resilience** - Provides better user experience during unstable network conditions

#### Implementation Notes

- Session resumption handles are managed internally by the ADK framework
- No additional client-side code changes are required
- The feature is particularly beneficial for long-running streaming conversations
- Connection interruptions become less disruptive to the user experience

#### Troubleshooting

If you encounter errors with session resumption:

1. **Check model compatibility** - Ensure you're using a model that supports session resumption
2. **API limitations** - Some session resumption features may not be available in all API versions
3. **Remove session resumption** - If issues persist, you can disable session resumption by removing the `session_resumption` parameter from `RunConfig`

#### Message Sending Endpoint

```py
Expand Down Expand Up @@ -427,7 +481,7 @@ async def send_message_endpoint(user_id: int, request: Request):
- **Error Handling** - Returns appropriate error responses for unsupported MIME types or missing sessions.


## 6. Client side code overview { #client-side-code-overview }
## 6. Client side code overview {#6.-client-side-code-overview}

The client-side consists of a web interface with real-time communication and audio capabilities:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
from google.adk.runners import InMemoryRunner
from google.adk.agents import LiveRequestQueue
from google.adk.agents.run_config import RunConfig
from google.genai import types

from fastapi import FastAPI, WebSocket
from fastapi.staticfiles import StaticFiles
Expand Down Expand Up @@ -66,7 +67,10 @@ async def start_agent_session(user_id, is_audio=False):

# Set response modality
modality = "AUDIO" if is_audio else "TEXT"
run_config = RunConfig(response_modalities=[modality])
run_config = RunConfig(
response_modalities=[modality],
session_resumption=types.SessionResumptionConfig()
)

# Create a LiveRequestQueue for this session
live_request_queue = LiveRequestQueue()
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
google-adk==1.9.0
google-adk==1.10.0
6 changes: 5 additions & 1 deletion examples/python/snippets/streaming/adk-streaming/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
from google.adk.runners import InMemoryRunner
from google.adk.agents import LiveRequestQueue
from google.adk.agents.run_config import RunConfig
from google.genai import types

from fastapi import FastAPI, Request
from fastapi.staticfiles import StaticFiles
Expand Down Expand Up @@ -66,7 +67,10 @@ async def start_agent_session(user_id, is_audio=False):

# Set response modality
modality = "AUDIO" if is_audio else "TEXT"
run_config = RunConfig(response_modalities=[modality])
run_config = RunConfig(
response_modalities=[modality],
session_resumption=types.SessionResumptionConfig()
)

# Create a LiveRequestQueue for this session
live_request_queue = LiveRequestQueue()
Expand Down
Original file line number Diff line number Diff line change
@@ -1 +1 @@
google-adk==1.2.1
google-adk==1.10.0