Skip to content

[BUG]: mTLS client certificate ignored unless a custom CA certificate is also configured #6771

Description

@Meekohi

🐞 Bug Summary

Outbound HTTP clients build an SSL context only when a custom CA certificate is configured. A gateway that requires mTLS but presents a server certificate chaining to the system trust store therefore never has its configured client_cert / client_key loaded into the httpx client, and the TLS handshake fails (or the peer rejects the connection for lacking a client certificate).

The gating is the same in five places:

if url.startswith("http://"):
    ctx = None
elif ca_certificate:                      # <-- client_cert/client_key ignored when falsy
    ctx = get_cached_ssl_context(ca_certificate, client_cert=..., client_key=...)
else:
    ctx = None                            # <-- client identity dropped
Location Path affected
mcpgateway/services/gateway_service.py:4805 gateway health check
mcpgateway/services/gateway_service.py:7276 connect_to_sse_server
mcpgateway/services/gateway_service.py:7447 connect_to_streamablehttp_server
mcpgateway/services/gateway_service.py:8416 test_gateway_handshake
mcpgateway/services/tool_service.py:6310 live tool invocation (MCP/SSE)

The underlying helper is already correct: get_cached_ssl_context(None, client_cert=..., client_key=...) keeps the system trust store for verification and still calls load_cert_chain (covered today by test_get_cached_ssl_context_with_none_ca_and_client_certs). oauth_manager.py:369 already gates correctly with if ca_certificate or client_cert or client_key:. Only these five call sites disagree.


🧩 Affected Component

  • mcpgateway - API
  • mcpgateway - UI (admin panel)
  • Federation or Transports
  • CLI, Makefiles, or shell scripts
  • Container setup (Docker/Podman/Compose)
  • Other (explain below)

🔁 Steps to Reproduce

  1. Stand up an MCP server over HTTPS whose server certificate is issued by a CA already in the system trust store (a public CA, or any CA installed in the container's bundle), and configure it with ssl_context.verify_mode = ssl.CERT_REQUIRED so it demands a client certificate.
  2. Register it as a gateway with client_cert and client_key set, and no ca_certificate (none is needed — the server cert already verifies).
  3. Let the health check run, or invoke a tool on that gateway.

Result: the handshake fails because no client certificate is presented; the gateway is marked unreachable and tool calls are refused.

Setting ca_certificate to the (redundant) issuing CA is the only workaround, because it flips the elif branch and pulls the client cert in as a side effect.


🤔 Expected Behavior

A configured client_cert / client_key pair should be presented to the peer whenever the connection is TLS, regardless of whether a custom CA certificate is also configured. Verification should continue to use the system trust store when no custom CA is supplied.


📓 Logs / Error Output

The defect is visible directly in code without needing a live peer: with ca_certificate unset,
get_cached_ssl_context is never called, so verify falls through to get_default_verify() — a
plain bool — and the client cert is never loaded. The exact alert the peer returns depends on its
TLS version and configuration, so I have not pasted a transcript rather than quote one I did not
capture on a real server.


🧠 Environment Info

Key Value
Version or commit main@f625315ea (also present in v1.0.10)
Runtime Python 3.12
Platform / OS Ubuntu 24.04
Container none (also reproduces in-container)

🧩 Additional Context

This is the residual half of #3579 (NET-01). PR #3758 fixed that report by threading client_cert/client_key into the cache key and the get_cached_ssl_context() calls, but left them inside the elif ca_certificate: branch — and #3579's repro generated its own CA, so the custom-CA path was the only one exercised. #3629 later added the http:// short-circuit, producing the current three-branch shape.

Related but distinct: #5854 (SSL_CERT_FILE support for custom CA bundles) touches the same module for a different reason.

I have a fix with regression tests ready and will open a PR referencing this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions