feat(genai): surface usage_metadata.traffic_type in response_metadata - #1948
Open
koenroelofs-taktile wants to merge 2 commits into
Open
feat(genai): surface usage_metadata.traffic_type in response_metadata#1948koenroelofs-taktile wants to merge 2 commits into
koenroelofs-taktile wants to merge 2 commits into
Conversation
On the Vertex AI backend, GenerateContentResponse.usage_metadata includes traffic_type (ON_DEMAND, ON_DEMAND_PRIORITY, ON_DEMAND_FLEX, PROVISIONED_THROUGHPUT), reporting which quota served the request. _response_to_result read token counts off that object but dropped this field, leaving no supported way to read it downstream. Copy it (as the enum name) onto response_metadata and generation_info, gated on finish_reason so it only appears on the final chunk when streaming - mirroring how model_name avoids duplication under chunk concatenation with +=. Fixes langchain-ai#1947 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ignment TrafficType is a CaseInSensitiveEnum that coerces even unknown API values to members with a .name, so the getattr/str fallback was dead code. Both generation_info and response_metadata assignments now live in one finish_reason-gated block (like grounding_metadata), and the class docstring's response metadata example documents the new key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
On the Vertex AI backend,
GenerateContentResponse.usage_metadataincludestraffic_type(ON_DEMAND,ON_DEMAND_PRIORITY,ON_DEMAND_FLEX,PROVISIONED_THROUGHPUT), reporting which quota served the request._response_to_resultreads token counts off that same object but drops this field, so there is no supported way to tell (e.g.) whether Priority Pay-as-you-go was granted for a given request. The raw response isn't kept on theChatResult, so the value is unrecoverable downstream without monkeypatching.This PR copies
usage_metadata.traffic_type(as the enum name) ontomessage.response_metadata["traffic_type"]andgeneration_info["traffic_type"].Areas for careful review: placement is gated on
candidate.finish_reasonso the value only appears on the final chunk when streaming — the same patternmodel_nameuses to avoid string duplication when chunks are concatenated with+=. When the backend doesn't report a traffic type (Gemini Developer API), the key is omitted entirely; an explicitTRAFFIC_TYPE_UNSPECIFIEDis passed through as-is rather than filtered.Relevant issues
Fixes #1947
Type
🆕 New Feature
Changes(optional)
_response_to_result: extracttraffic_typefromresponse.usage_metadata(via.name) and attach it togeneration_infoandresponse_metadatain a singlefinish_reason-gated block (final chunk only).traffic_typeadded to the response metadata example (noted as Vertex-only).+=concatenation).Testing(optional)
make test: 385 passedmake lint: ruff format, ruff check, and mypy all cleanNote(optional)
Disclaimer: this contribution was developed with the assistance of an AI agent (Claude Code), reviewed and directed by the issue author.
🤖 Generated with Claude Code