Skip to content

fix: avoid replaying transient Responses API items through PatchedChatOpenAI#1455

Open
supper-ming wants to merge 1 commit intobytedance:mainfrom
supper-ming:fix-responses-store-replay
Open

fix: avoid replaying transient Responses API items through PatchedChatOpenAI#1455
supper-ming wants to merge 1 commit intobytedance:mainfrom
supper-ming:fix-responses-store-replay

Conversation

@supper-ming
Copy link
Copy Markdown

Summary

This PR makes PatchedChatOpenAI safe to use with OpenAI-compatible Responses API gateways that do not persist response items when store=false.
Changes:

  • keep the existing Gemini thought_signature replay fix
  • strip non-replayable Responses API items such as reasoning blocks from replayed
    history when store is not enabled
  • preserve replayable tool-call items like function_call and function_call_output
  • add regression tests for both store=false and store=true
  • clarify the configuration docs for OpenAI-compatible Responses API gateways

Root Cause

In multi-turn conversations, DeerFlow replays prior assistant history back into the next
Responses API request.

For some OpenAI-compatible gateways, when store=false, response-only items such as
reasoning are not persisted server-side. Replaying those opaque items in the next turn
can fail with errors like:

Item with id 'rs_...' not found. Items are not persisted when `store` is set to false.

This surfaced in DeerFlow after clarification/tool-driven turns, where LangChain forwarded
prior Responses API items back into input unchanged.

## Why this fix

PatchedChatOpenAI already exists to handle OpenAI-compatible gateway quirks, specifically
Gemini thought_signature replay. Extending it to also sanitize non-replayable Responses
API items keeps the fix localized to the compatibility layer and avoids changing normal
thread or frontend behavior.

The patch:

- removes transient response items only when store is not explicitly enabled
- keeps tool-call continuity intact
- preserves existing behavior when store=true

## Tests

Ran:

uv run --project backend --group dev python -m pytest backend/tests/test_patched_openai.py
-q

Result:

10 passed

## Notes

This PR intentionally does not include:

- local environment/port changes
- frontend thread/navigation changes

It is scoped only to the confirmed backend compatibility issue.

@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants