Add bedrock-mantle support to chat_aws_bedrock() - #1064
Open
hadley wants to merge 9 commits into
Open
Conversation
Bedrock serves models from two endpoints. bedrock-mantle speaks the Anthropic Messages and OpenAI Responses APIs, and hosts models that the Converse API on bedrock-runtime can't reach at all, including Claude Mythos and the GPT-5 family. chat_aws_bedrock() gains an `api` argument to pick between them, using ProviderAnthropic and ProviderOpenAI subclasses that swap the API key for the existing paws SigV4 flow. The API is guessed from `model` by default, falling back to converse, so existing code is unaffected. The model -> api lookup is derived from litellm by data-raw/prices.R and written to R/provider-aws-api.R, so it refreshes with the weekly prices job. It only lists models converse can't serve, which keeps a data refresh from moving a working call onto a different endpoint. Also extracts the Anthropic and OpenAI error body parsers so the mantle providers can fall back to the AWS error shape, which is what surfaces for signature, credential, and throttling failures.
Live testing showed frontier models reject /v1/responses: mantle serves newer models from /openai/v1 and older open-weight ones like gpt-oss from /v1, and the two are disjoint rather than aliases. Confirmed against gpt-5.4, gpt-5.6-sol, grok-4.3, gemma-4, and gpt-oss-20b, and documented in the model cards. Everything we route to responses is on /openai/v1, since the /v1 models are all reachable through converse anyway. Also adds the error hint the plan called for: a model that only exists on mantle otherwise fails against converse with a bare "The provided model identifier is invalid", which gives no clue that `api` is the fix.
Member
Author
|
@gadenbuie mostly an FYI so you can see that I'm using the liteLLM data in another way |
The model now determines the API rather than the reverse, so there's a single default model again instead of one per API. That removes the ordering dependency where the API had to be resolved before the model. Deletes the base_url closures: aws_bedrock_base_url() takes the region directly, and models_aws_bedrock() no longer special-cases the control plane URL because models_list() already rewrites the host. Two bugs found while doing this: * aws_error_body() called `if (resp_content_type(resp) != ...)`, which errors when a response has no content type at all, masking the real error with "Failed to parse error body". * models_aws_bedrock() derived the listing URL from the chat base_url, so it broke for the responses API when that moved to /openai/v1. Mantle lists every model at one /v1/models regardless of which API you chat with, so build that URL from the region instead.
The three constructor calls differed only in the class and how each expresses caching, so the other nine arguments were written out three times. aws_bedrock_class() maps the api to a class name, get() resolves it, and the cache argument is spliced in.
thisisnic
reviewed
Jul 27, 2026
Co-authored-by: Nic Crane <thisisnic@gmail.com>
Regenerated by the pricing workflow; keep the binary out of this PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AWS now serves Bedrock models from two endpoints. The original
bedrock-runtimeendpoint speaks the Converse API, which is what ellmer has always used. The newerbedrock-mantleendpoint speaks the Anthropic Messages and OpenAI Responses APIs instead, and hosts models that Converse can't reach at all — including the GPT-5 family, Grok 4.3, and Claude Mythos. Until now those models were simply unavailable through ellmer.chat_aws_bedrock()gains anapiargument to choose between them. You rarely need to set it: ellmer picks the API from the model name and falls back to Converse for anything it doesn't recognise, so existing code behaves exactly as before.Example
Passing a mantle-only model without an
apiused to fail with a bare "The provided model identifier is invalid". That error now suggests the fix.Design decisions
The argument names the request format, not the endpoint.
apitakes"converse","messages", or"responses"rather than"runtime"and"mantle", because one mantle endpoint serves three different request formats — naming the endpoint wouldn't say which one to send.Chat Completions is deliberately not supported. Every Bedrock model that speaks it also speaks either Converse or Responses, so it would add no model coverage.
The model lookup is generated, not hand-written.
data-raw/prices.Ralready fetches litellm's model metadata weekly, and that data records which models live only on mantle. It now also writesR/provider-aws-api.R, so the table refreshes on the existing schedule. It's a plain R file rather than binary data insysdata.rdaso the weekly diff is reviewable.The table only lists models Converse can't serve. Everything else falls through to Converse. This keeps a data refresh from silently moving a working call onto a different endpoint with different quotas — it can only ever turn a failure into a success. It also means an unfamiliar model id (a custom ARN, a new inference profile) keeps working as it does today rather than being rejected.
Not included
token_usage()reportsNAfor them, including on mantle. That's a pre-existing bug in the same extraction script, tracked in Bedrock prices missing for all modern Claude models #1063.Verification
Live-tested against a real AWS account: basic chat, streaming, tool calling, and structured data all work on
openai.gpt-5.6-solvia"responses", and onanthropic.claude-haiku-4-5via"messages".That testing caught a bug worth flagging for review: mantle has two OpenAI-compatible paths that are disjoint rather than aliases. Newer models are served from
/openai/v1and older open-weight ones like gpt-oss from/v1, and each rejects the other's models. AWS's API compatibility tables don't record this distinction — only the individual model cards mention it — so the first implementation used/v1and failed on exactly the models this PR is meant to unlock.Note that
test-provider-aws.R:229currently fails. It's unrelated to this PR — it fails identically onmain, and was previously invisible because it skips without AWS credentials.🤖 Generated with Claude Code