Chore/mcp sdk v2 - #6787
Open
madhu-mohan-jaishankar wants to merge 3 commits into
Open
Conversation
* chore(deps): drop fastmcp dev dependency and protocol-compliance make targets Signed-off-by: Jonathan Springer <jps@s390x.com> * test(live-gateway): rewrite MCP protocol E2E on the official mcp SDK Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(tests): remove the compliance reference server and matrix tooling The fastmcp-based protocol-compliance harness was the only consumer of the compliance reference server and the compliance matrix generator, so both go with it. - delete mcp-servers/python/compliance_reference_server/ and scripts/compliance_matrix.py - drop the compliance-reference-server dev dependency and its [tool.uv.sources] editable entry; regenerate uv.lock (also sheds fastmcp, fastmcp-slim, authlib and their transitives) - drop the compliance-only pytest markers from pyproject.toml - refresh the pip-audit skip-list note and the PYTEST_IGNORE comment Signed-off-by: Jonathan Springer <jps@s390x.com> * docs: drop protocol-compliance references and name the mcp SDK client Follow-up to the harness removal: the PR-review validation gate, the live-gateway README/target list, and the test docs still pointed at `make test-protocol-compliance` and described the protocol E2E suites as FastMCP-driven. - AGENTS.md, tests/AGENTS.md, llms/NOTES.template.md, docs/docs/development/release-management.md, tests/live_gateway/README.md: retire the removed targets, name the official mcp SDK client - tests/live_gateway/helpers/mcp_test_helpers.py: correct the module docstring's consumer list and point new tests at the SDK ClientSession pattern Signed-off-by: Jonathan Springer <jps@s390x.com> * test(sso): move Entra ID integration test to tests/integration Ports the relocation from mcp-2-0-0-prerelease (d7c1f9a): the Entra ID suite is fully in-process (httpx.ASGITransport) and self-skips when the AZURE_* credentials are absent, so it belongs with the integration tests that run under the default 'make test' rather than the live-gateway tree. - tests/live_gateway/sso/test_entra_id_integration.py -> tests/integration/test_entra_id_integration.py (header updated) - tests/live_gateway/sso/__init__.py: docstring now covers only the Keycloak suite and points at the new Entra ID location - docs/docs/testing/entra-id-e2e.md: all pytest invocation paths updated - Makefile: test-e2e-sso targets Keycloak only, notes the Entra ID move (help comment aligned as well) Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(deps): remove fastmcp and rebuild live-gateway E2E on the official mcp SDK Drops the fastmcp dev dependency and everything that depended on it, replacing the live-gateway protocol and RBAC E2E suites with a client built on the official mcp Python SDK. - drop fastmcp dev dependency and protocol-compliance make targets - remove the fastmcp-based protocol compliance harness and the fast-server go/rust parity test - rewrite the MCP protocol and RBAC transport E2E suites on the official mcp SDK ClientSession - remove the compliance reference server and compliance matrix tooling (the harness was their only consumer); regenerate uv.lock, which also sheds fastmcp-slim, authlib and their transitives - update AGENTS.md, test docs, and the live-gateway README to retire the removed targets and name the mcp SDK client Closes #5910 Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(deps): drop fastmcp dev dependency and protocol-compliance make targets Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(tests): remove the compliance reference server and matrix tooling The fastmcp-based protocol-compliance harness was the only consumer of the compliance reference server and the compliance matrix generator, so both go with it. - delete mcp-servers/python/compliance_reference_server/ and scripts/compliance_matrix.py - drop the compliance-reference-server dev dependency and its [tool.uv.sources] editable entry; regenerate uv.lock (also sheds fastmcp, fastmcp-slim, authlib and their transitives) - drop the compliance-only pytest markers from pyproject.toml - refresh the pip-audit skip-list note and the PYTEST_IGNORE comment Signed-off-by: Jonathan Springer <jps@s390x.com> * docs: drop protocol-compliance references and name the mcp SDK client Follow-up to the harness removal: the PR-review validation gate, the live-gateway README/target list, and the test docs still pointed at `make test-protocol-compliance` and described the protocol E2E suites as FastMCP-driven. - AGENTS.md, tests/AGENTS.md, llms/NOTES.template.md, docs/docs/development/release-management.md, tests/live_gateway/README.md: retire the removed targets, name the official mcp SDK client - tests/live_gateway/helpers/mcp_test_helpers.py: correct the module docstring's consumer list and point new tests at the SDK ClientSession pattern Signed-off-by: Jonathan Springer <jps@s390x.com> * feat: migrate to MCP SDK 2.0.0b2 with modern 2026-07-28 upstream negotiation Squashed arc: - MCP SDK migration 2.0.0a2 -> 2.0.0b1 -> 2.0.0b2 (Python 3.14 support added; floor stays >=3.11 — mcp 2.0.0b2 needs only >=3.10, cpex >=3.11, CI is py3.12, MCPError rename, transport/Client compatibility fixes) - Upstream federation path migrated from raw mcp.ClientSession to mcp.client.Client: mode=auto probes server/discover and adopts modern 2026-07-28, with transparent legacy fallback; MCP_CLIENT_CONNECT_MODE=legacy rollback flag - Pooled upstream session registry on the high-level Client with typed SessionLifecycle handle (replaces _cf_* attribute smuggling), cache=False to preserve ADR-052 notification wiring, dispatcher-aware transport health check - mcp_proxy_client gained transport= (streamablehttp/sse) and mode= options; all five ad-hoc SSE fallback sites routed through it - Compose test stack: single cfex-mcp-fast-time-server image (stale Go-era flags fixed), strict modern-only 2026-07-28 fast_time_2026 variant added, five register_* one-shots consolidated into a single 'register' container running scripts/register_mcp_servers.py 'stack' - Tests: SDK invariant spike suite, config validators, reworked registry/service suites, E2E negotiation suite (11 tests) proving auto/legacy modes against strict and legacy upstreams - Docs: MCP_CLIENT_CONNECT_MODE and legacy rollback documented - Fixes: four-parameter httpx2.Timeout in proxy-client default path, token-exchange integration test isolation from shared Redis, pre-existing get_httpx_client_factory dedent repair in gateway_service Packaging — the SDK pin lands in core dependencies, not in extras: - An earlier revision of this branch split the `mcp` pin into mutually exclusive `runtime` / `live-tests` extras backed by a `[tool.uv] conflicts` block, so the gateway could stay on `mcp` 2.x while the fastmcp-based protocol-compliance harness stayed on `mcp` < 2.0. That harness is gone, so the partition is removed: `mcp==2.0.0b2` and `mcp-types==2.0.0b2` sit in core `dependencies` alongside `cpex`, and `pip install mcp-contextforge-gateway` once again resolves a working install. - Every `--extra runtime` / `--extra live-tests` flag is dropped from the Make targets, the pytest/pytest-rust/wrapper workflows and both Containerfiles, restoring the install targets to their pre-extras form. The `granian` extra is dropped from the Containerfile install lists as well; no such extra has ever been defined. - All install documentation, the four affected ADRs and the agent-facing dependency-layout guidance revert to their extras-free wording. - Fixes a duplicate `cpex-secrets-detection` key in `[tool.uv.exclude-newer-package]` that left pyproject.toml invalid TOML. Also realigns two upstream session-factory tests that were left on the pre-migration SDK shape: they patched `streamablehttp_client` / `ClientSession` and reached for the retired `_cf_shutdown_event` / `_cf_owner_task` attributes, so they errored during setup instead of exercising the post-ready WARNING path and the timeout structured-logging fallback. Signed-off-by: Jonathan Springer <jps@s390x.com> * feat: upgrade MCP SDK from 2.0.0b2 to 2.0.0 final The 2.0.0 final release removes mcp.shared.session entirely (the b2 typing-only RequestResponder stub is gone) and changes Client(cache=) from a bool flag to CacheConfig | None, where None disables the response cache so message_handler passes through unwrapped. - pyproject/uv.lock: mcp/mcp-types >=2.0.0, cpex tracked from the 0.1.x-mcp-2.0.0 branch, drop stale exclude-newer pins - mcpgateway/utils/session_compat.py: new shim re-creating the RequestResponder typing stub so existing annotations and isinstance guards keep working (never instantiated, matching b2 semantics) - notification_service/upstream_session_registry: import from the shim; pass cache=None (was cache=False) in the pooled upstream session factory so the response cache does not wrap message_handler - tests: repoint RequestResponder imports to the shim, assert cache is None, update gateway refresh test to the mcp_proxy_client seam, refresh upstream session error category expectations, regenerate .secrets.baseline Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(services): align MCP httpx client factories on httpx2 types Signed-off-by: Jonathan Springer <jps@s390x.com> * test(live-gateway): migrate MCP e2e suites to SDK 2.0 final client API Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(compose): restore per-service REST auto-registration, drop strict 2026-07-28 probe Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(testing): drop stale Fast Time 2026 strict-probe references The fast_time_2026_server service was removed from docker-compose.yml in 5e54707, but its advertising and test wiring lingered: - scripts/register_mcp_servers.py: delete (unused anywhere in the repo) - Makefile: drop the :8887 banner and auto-registration echo from testing-up - tests/e2e/test_upstream_connect_mode_e2e.py: strip the strict-upstream behavior matrix, wire-capture proofs, and strict registration tests; keep the legacy connect-mode proofs against fast_time_server (:8888) Signed-off-by: Jonathan Springer <jps@s390x.com> * Restore Containerfile and Makefile to main state Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix(mcp-2.0): address SDK 2.0 review findings and live-stack build issues Review findings (PR #5493): - transports/streamablehttp: ctx.meta is a TypedDict (RequestParamsMeta) in MCP 2.0 with no model_dump(); extract via dict() with a legacy-model fallback at the call_tool/get_prompt/read_resource sites - transports/streamablehttp: complete() now returns types.CompleteResult wrapping types.Completion so the wire shape carries the required 'completion' nesting; update test assertions to the corrected contract - utils/streamable_http_compat: build httpx2.AsyncClient (not v0 httpx) in the no-factory fallback, matching what the MCP 2.0 SDK transports expect; same-class fixes in resource_service factory and the upstream session registry (HttpxClientFactory alias + httpx/httpx2 exception matching) - services/http_client_service: add get_httpx2_timeout(); the v0 httpx.Timeout from get_http_timeout() is rejected by httpx2.AsyncClient, so point the gateway/tool/resource factory fallbacks at the new helper - transports/streamablehttp: capture server-emitted session IDs whenever stateful sessions are enabled, not only when multi-worker affinity is on, so the logical-owner claim fires on single-node deployments; unskip the affinity-disabled regression test Live stack: - Containerfile: add git to the builder stage (uv needs it to fetch the git-pinned cpex dependency) and split the monolithic venv RUN into discrete steps so build failures name the exact stage - docker-compose: repin cfex-mcp-fast-time-server to the current digest; the previous pin predates the 2026-07-28 caching directives and its tools/list responses failed ListToolsResult validation during federation Also rewrite test_invoke_tool_mcp_refuses_stored_env_filter to the v2 mcp_proxy_client mock pattern (pre-existing stale migration leftover). Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: direct_proxy resource reads — SSE transport, final content shape, blob decoding Three defects left direct_proxy federated resource reads broken or silently corrupting content: 1. mcp_proxy_client was called without a transport argument, so SSE gateways were dialed with streamable-HTTP framing and the connect failed. Select the transport from the gateway's registered type. 2. The proxied read already returns final content, but execution fell through into the cache-mode pointer-resolution machinery, which assumes a DB-shaped object and crashed on content.id (AttributeError, masked to "" by the ingress). Build the final ResourceContent (with id and mime_type) in the branch and skip the machinery via a flag initialized at function scope alongside the other locals. 3. MCP transports carry blobs as base64 strings while ResourceContent.blob holds raw bytes; passing the string through meant the ingress re-encoded it, double-encoding every binary resource. Decode to raw bytes at the seam. Tests: TestReadResourceDirectProxy updated to assert the final content shape (ResourceContent) — the previously asserted intermediate shape is exactly what crashed the pointer-resolution machinery in production — and the blob test now uses valid base64 and asserts decoded raw bytes. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: let 2026-era requests through the session-less peeks so the ingress can serve modern sessions The streamable ingress wrapper peeks at session-less POSTs to give unknown-method probes a truthful -32601 (instead of the SDK's 'Missing session ID'). But 2026-era requests are self-describing and session-less BY DESIGN, and server/discover is not in the gateway's known-method list — so every modern client's discover probe was answered -32601 here, before the SDK's era routing ever ran, and the client silently fell back to the legacy handshake. The gateway could consume modern upstream sessions but never serve one. Mirror the SDK's own era-dispatch rule (MCP-Protocol-Version header present and not a handshake version => modern) and skip both session-less peeks for such requests: the SDK's modern classifier owns their validation and rejections. Also add server/discover to the known-method list. With this, a v2 client negotiates 2026-07-28 through the gateway (tools, prompts, and resources verified against both a 1.x-era SSE downstream and a 2026-era streamable downstream); legacy clients are unaffected (no header / handshake header keeps the old peek behavior, pinned by test). Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * refactor(tests): group era-aware peek-bypass tests under TestModernEraPeekBypass Groups the three behaviors pinning the modern-era peek bypass (modern header passes through, handshake header still peeked, server/discover is a known method) under one class with a shared docstring telling the era-boundary story — easier to locate and read as a unit in this large test module. No behavior change. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * rename era peek-bypass group to TestDualEra Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: standing listen streams for modern servers feeding debounced refresh Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * refactor: module-level listener imports, tighter notify_list_changed docstring Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: prune stale catalog rows created via notification-driven refresh Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: answer unknown tool/prompt/resource requests in proper error shapes for both eras Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: relay upstream tool progress to downstream callers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: surface tool invocation failures as isError results for both eras Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: pass through 2026 MRTR elicitation for modern clients on modern servers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * port: move #5934 MCP handshake probe onto the SDK 2.0 client The Test Connection endpoint's initialize-path probe was written against the SDK 1.x client API. Port it to SDK 2.0: _SniPinningTransport is now built on httpx2 (the stack the v2 client transports run on) with identical pinning logic, the local pinned-client factory returns httpx2.AsyncClient, _probe_session reads the v2 result field names, and the probe connects via the v2 sse_client / streamable_http_client. The raw server/discover probe path is unchanged. Tests updated to the v2 shapes; every pinning assertion (sni_hostname, pinned_host, TLS verify context) still asserts. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Co-authored-by: Jonathan Springer <jps@s390x.com>
* ci: run PR checks on pull requests targeting chore/mcp-sdk-v2
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: relay plugin-blocked tool calls as isError results on the /mcp path
A PluginViolationError raised on the SDK-served /servers/{id}/mcp path fell
through to the generic handler and was wrapped as a bare code-0 JSON-RPC
error. Catch it in call_tool and return an isError CallToolResult, matching
the contract the plugin e2e suites assert.
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: port inherited unit tests to SDK v2 call shapes
The tests still targeted the SDK 1.x API: the streamablehttp_client patch
target and 3-tuple transport, camelCase result fields, session.call_tool
mocks where v2 calls client.session.call_tool, and call_tool assertions
without the v2 keyword arguments. Header assertions now inspect the
http_client passed to the SDK instead of a headers kwarg.
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: satisfy pylint and pre-commit on migration files
Disable no-member on the mcp_app.request_context reads (the property is
injected at runtime for the v1 handler adapters) and try-except-raise on
the listener's CancelledError guard; strip a trailing space flagged by
pre-commit.
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: satisfy pre-commit and diff coverage for the plugin-block change
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: remove unwanted commentsn from tests
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
---------
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…release verification (#6716) * feat(compose): add 'ui' profile for contextforge-web-ui and document release verification Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix: lint issue Signed-off-by: Marek Dano <mk.dano@gmail.com> * docs: fix forbid-source-placeholders hook failure in ui.md Replace '(source: ...)' pattern on line 130 with an inline link so the pre-commit forbid-source-placeholders hook no longer rejects the file. Also refresh .secrets.baseline line numbers after the one-line change. Closes #6716 Signed-off-by: Marek Dano <mk.dano@gmail.com> * chore: refresh .secrets.baseline line numbers after rebase onto chore/mcp-sdk-v2 Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix(compose): pin web_ui image to a released version and clarify BFF UI docs Address PR review feedback: pin WEB_UI_IMAGE to a specific released tag instead of `latest`, add a release-checklist step to bump and verify that version, and clarify that the built-in UI and the BFF-style contextforge-web-ui are both supported side by side. Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix(compose): pin web_ui image by digest and add ui profile config check Pin WEB_UI_IMAGE to tag@digest (docker-compose.yml, .env.example) so the supported ui profile's session/auth-handling BFF service can't be silently swapped by a retargeted registry tag. Document the resolve-and-bump-digest step in release-management.md #6.4, add `make compose-ui-config-check` (docker compose --profile ui config --quiet) plus a matching CI workflow to catch profile/schema regressions on PRs, and drop the duplicated WEB_UI_IMAGE version literal from AGENTS.md so there's one source of truth. Addresses review feedback from vishu-bh and gcgoncalves on #6716. Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix: refresh .secrets.baseline line numbers for docker-compose.yml The baseline committed in ef347b9 was generated before the last edit to docker-compose.yml's web_ui image comment block, leaving it 5 lines stale and failing detect-secrets in CI (Pre-commit Checks, run 34468285949) even though it passed locally against the pre-edit state. Signed-off-by: Marek Dano <mk.dano@gmail.com> --------- Signed-off-by: Marek Dano <mk.dano@gmail.com>
madhu-mohan-jaishankar
requested review from
Lang-Akshay,
brian-hussey,
crivetimihai,
ja8zyjits and
msureshkumar88
as code owners
September 11, 2026 13:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.