[Bug]: 大数据量 MCP 请求（超过 4KB）时，网关/服务端因 UTF-8 截断导致 500 错误

[qw918]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

问题描述：
在 AI 客户端通过 LiteLLM 调用下游 MCP 服务端 https://xxx/mcp 时，若请求 Payload（如 HTML 报表）较大（超过 4096 字节），下游会频繁返回 500 Internal Server Error 导致调用失败。

报错日志（LiteLLM 侧记录的下游 500）：

litellm-1  | 11:34:32 - LiteLLM:ERROR: server.py:4022 - Error handling MCP request: 'utf-8' codec can't decode byte 0xe9 in position 4095: unexpected end of data
litellm-1  | Traceback (most recent call last):
litellm-1  |   File "/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/server.py", line 3811, in handle_streamable_http_mcp
litellm-1  |     is_initialize = _is_initialize_request(body)
litellm-1  |   File "/app/.venv/lib/python3.13/site-packages/litellm/proxy/_experimental/mcp_server/server.py", line 3196, in _is_initialize_request
litellm-1  |     data = json.loads(body)
litellm-1  |   File "/usr/lib/python3.13/json/__init__.py", line 347, in loads
litellm-1  |     s = s.decode(detect_encoding(s), 'surrogatepass')
litellm-1  | UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 4095: unexpected end of data
litellm-1  | INFO:     172.121.12.23:56866 - "POST /preadvice/mcp HTTP/1.1" 500 Internal Server Error

我们目前的临时规避手段（客户端 Patch）： 我们在 AI 客户端代码中加上了重试机制。当遇到 500 错误时，如果在重试中向 JSON 请求体最前面填充1~3个空格（不破坏 JSON 语法，但能偏移物理截断边界，使 4096 字节处对齐到合法字符边界），下游就能解码成功并返回 200 OK。但这属于客户端的被动规避，会额外增加一次 RTT（往返延迟）。

Steps to Reproduce

1. 构造一个特殊的 Payload，使其前 4095 字节全部为 ASCII 字符（单字节），第 4096 字节正好是中文字符（3 字节）的第 1 个字节。 当网关截取前 4096 字节时，末尾会残存一个孤立的 UTF-8 引导字节（例如 \xe6），导致 UTF-8 解码失败，网关直接返回 500。

Relevant log output

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v.1.90.0

Twitter / LinkedIn details

No response

[Jupiter363]

I was able to reproduce this in the streamable HTTP routing peek. The 4096-byte body peek is only meant to classify initialize requests, but json.loads(body) can raise UnicodeDecodeError when the peek ends inside a multibyte UTF-8 sequence. That sends the request to the generic 500 handler before the downstream MCP handler can receive the full body

I have a small fix ready that treats this the same way as other unparseable peek data: keep the existing peek cap, avoid buffering or draining the whole payload for routing, replay the consumed ASGI messages, and let non-initialize requests continue to the stateless handler. I also handled the adjacent JSON-RPC response peek path with the same decode behavior

The regression test splits a multibyte character exactly at the 4096-byte boundary and asserts the full original body still reaches the downstream handler without emitting a 500

[qw918]

I was able to reproduce this in the streamable HTTP routing peek. The 4096-byte body peek is only meant to classify initialize requests, but json.loads(body) can raise UnicodeDecodeError when the peek ends inside a multibyte UTF-8 sequence. That sends the request to the generic 500 handler before the downstream MCP handler can receive the full body

I have a small fix ready that treats this the same way as other unparseable peek data: keep the existing peek cap, avoid buffering or draining the whole payload for routing, replay the consumed ASGI messages, and let non-initialize requests continue to the stateless handler. I also handled the adjacent JSON-RPC response peek path with the same decode behavior

The regression test splits a multibyte character exactly at the 4096-byte boundary and asserts the full original body still reaches the downstream handler without emitting a 500

Thanks a lot for the quick turnaround and the elegant fix!