Fix Structured Output for GPT-OSS Models

[windreamer]

Motivation

GPT-OSS models use Harmony Response format, which conflicts with Guided Decoding (token-level JSON constraint) when response_format is specified. This causes:

• Harmony parse errors
• Request hangs
• Empty message.parsed results

Breaking existing OpenAI SDK clients using client.beta.chat.completions.parse().

Modification

Approach: Replace Guided Decoding with Harmony-native structured output.

1. Detect GPT-OSS architecture with active response_format
2. Inject JSON schema into system message under # Response Formats section
3. Disable Guided Decoding by clearing response_format
4. Create system message automatically if none exists

---

closes: #4347

[Copilot]

Pull request overview

This PR fixes structured output for GPT-OSS models by avoiding Guided Decoding (which conflicts with Harmony response parsing) and instead injecting the requested response schema into the prompt using Harmony’s native # Response Formats section.

Changes:

• Detect GPT-OSS (arch == 'GptOssForCausalLM') requests with non-text response_format.
• Inject the serialized response_format schema into the system message under # Response Formats (creating a system message if missing).
• Disable guided decoding for GPT-OSS by clearing the local response_format passed into GenerationConfig.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jingyibo123]

It's been a while since I compiled from source ,does CUDA 12.1 + GCC 9.4 work?

[windreamer]

It's been a while since I compiled from source ,does CUDA 12.1 + GCC 9.4 work?

No need to recompile it, you can just patch the python part.

[jingyibo123]

After patching this