botocore: add basic tracing for bedrock InvokeAgent by stanek-michal · Pull Request #3404 · open-telemetry/opentelemetry-python-contrib

stanek-michal · 2025-04-03T03:51:38Z

Description

Add basic support for InvokeAgent API. For now only returnControl function invocation, non-streaming mode. Assumes the agent was created beforehand in Bedrock platform - proper Agent creation/deletion for testcases will be added in another PR.

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

tox -e py310-test-instrumentation-botocore

Does This PR Require a Core Repo Change?

Yes. - Link to PR:
No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

Followed the style guidelines of this project
[] Changelogs have been updated
Unit tests have been added
Documentation has been updated

linux-foundation-easycla · 2025-04-03T03:51:42Z

The committers listed above are authorized under a signed CLA.

✅ login: stanek-michal / name: Michal Stanek (4c4c419, 360d048, 01521d5)

xrmx

Haven't looked at the actual implementation yet, in the meantime a few random comments. Also it looks like you don't have pre-commit setup, see CONTRIBUTING.

xrmx · 2025-04-03T07:52:30Z

+def test_invoke_agent_with_completion(
+    span_exporter,
+    metric_reader,
+    bedrock_runtime_client,


Do you still need the bedrock runtime client?

no, though the create_agent scaffolding will need it still (for s3 and such) but it will be done elsewhere

xrmx · 2025-04-03T07:55:48Z

+        self.client = boto3.client('bedrock-agent-runtime')
+
+        # Replace these with your actual agent IDs
+        self.agent_id = "TFMZVIWXR7"


Let's use env vars then

changed it on both standalone demo and tests
(so tests now need export TOX_OVERRIDE=testenv.pass_env=AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_SESSION_TOKEN,AWS_DEFAULT_REGION,BEDROCK_AGENT_ID,BEDROCK_AGENT_ALIAS_ID

xrmx · 2025-04-03T07:59:59Z

@stanek-michal please sign the CLA!

estolfo · 2025-04-07T10:24:37Z

+                event_stream = result["completion"]
+
+                # Drain the stream so we can instrument AND keep events
+                all_events = list(event_stream)


Is there a reason we don't want to follow the same approach as we do with InvokeModelWithResponseStream?
It looks like we use a response stream wrapper that is an iterator.
I wonder what memory implications draining the stream all at once would have?

I thought it was simpler but you're right, apart of memory use I'm worried about introducing additional latency. For enableTrace=False there is just one event in my tests, but for enabled tracing there can be quite a few of them, so it matters. I'll change it to a wrapper but not sure if in this PR or next one with create agent

estolfo · 2025-04-07T10:28:22Z

+                all_events = list(event_stream)
+
+                # A replay generator so user code can still iterate
+                result["completion"] = _replay_events(all_events)


if we use a wrapper that's an iterator, as mentioned above, I'm assuming we wouldn't need this line

xrmx · 2025-04-28T12:53:48Z

-                span.update_name(f"{operation_name} {request_model}")
+            if self._call_context.operation == "InvokeAgent":
+                if operation_name:
+                    span.update_name(f"{operation_name}")


The **span name** SHOULD be `invoke_agent {gen_ai.agent.name}` if `gen_ai.agent.name` is readily available.

Do we have the agent name at hand?

It looks like InvokeAgent doesn't specify the agent name in request or response, just agentAlias. Seems like a mandatory step to set up an alias in order to call InvokeAgent API (according to my understanding of 1 and 2):

To deploy your agent, you must create an alias. During alias creation, Amazon Bedrock creates a version of your agent automatically. The alias points to this newly created version. Alternatively, you can point the alias to a previously created version of your agent. Then, you configure your application to make API calls to that alias.

Since it's a required request param, I think we can use it here.

xrmx · 2025-04-28T13:11:48Z

+                    api_input = input_item["apiInvocationInput"]
+                    action_group = api_input.get("actionGroup")
+                    span.set_attribute(GEN_AI_TOOL_NAME, action_group)
+                    span.set_attribute(GEN_AI_TOOL_TYPE, "extension")


Is this called directly by the agent though? From the examples here it looks like these are still called from client side https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeAgent.html#API_agent-runtime_InvokeAgent_ResponseSyntax

Semconv:

Extension: A tool executed on the agent-side to directly call external APIs, bridging the gap between the agent and real-world systems. Agent-side operations involve actions that are performed by the agent on the server or within the agent's controlled environment.

apiInvocation is untested for this PR but that's correct - if there is returnControl that means client executes the tool, not the Agent, so in this case it should be function not extension (but overall Agent has the capability to call lambdas on its own if it's configured for it)

SrdjanLL · 2025-04-28T14:31:53Z

-                span.update_name(f"{operation_name} {request_model}")
+            if self._call_context.operation == "InvokeAgent":
+                if operation_name:
+                    span.update_name(f"{operation_name}")


It looks like InvokeAgent doesn't specify the agent name in request or response, just agentAlias. Seems like a mandatory step to set up an alias in order to call InvokeAgent API (according to my understanding of 1 and 2):

To deploy your agent, you must create an alias. During alias creation, Amazon Bedrock creates a version of your agent automatically. The alias points to this newly created version. Alternatively, you can point the alias to a previously created version of your agent. Then, you configure your application to make API calls to that alias.

Since it's a required request param, I think we can use it here.

SrdjanLL · 2025-04-28T14:53:58Z



 _KNOWN_EXTENSIONS = {
+    "bedrock-agent-runtime": _lazy_load(


Proposal that I don't think is needed for this PR, but more of a future suggestion (and up for discussion):

It might be worth building a separate extension for bedrock-agent-runtime. So far, there seems to be a separate extension for every distinct botocore client (dynamodb, lambda, etc.) and bedrock-runtime is separate from bedrock-agent-runtime. Perhaps it's worth following that pattern and avoid some of the branching logic that's likely to grow as more Bedrock APIs get added.

Furthermore, to introduce agent creation/deletion means adding bedrock-agent client as well, which may again result in another extension or combining it with bedrock-agent-runtime and having one dedicated just to agents.

github-actions · 2026-03-11T03:56:30Z

This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment.
If you're still working on this, please add a comment or push new commits.

github-actions · 2026-03-25T04:04:00Z

This PR has been closed due to inactivity. Please reopen if you would like to continue working on it.

stanek-michal force-pushed the invoke-agent branch from 3ca64a2 to a193a11 Compare April 3, 2025 03:56

xrmx reviewed Apr 3, 2025

View reviewed changes

xrmx added this to Python PR digest Apr 3, 2025

xrmx moved this to Reviewed PR that needs fixing in Python PR digest Apr 3, 2025

estolfo reviewed Apr 7, 2025

View reviewed changes

xrmx reviewed Apr 11, 2025

View reviewed changes

Comment thread ...ry-instrumentation-botocore/src/opentelemetry/instrumentation/botocore/extensions/bedrock.py Outdated

stanek-michal added 2 commits April 17, 2025 13:20

Add initial version of invoke_agent tracing instrumentation

360d048

Add standalone invoke_agent instrumentation demo

01521d5

stanek-michal force-pushed the invoke-agent branch from a193a11 to f4e876d Compare April 17, 2025 11:20

Address review comments

4c4c419

stanek-michal force-pushed the invoke-agent branch from f4e876d to 4c4c419 Compare April 17, 2025 11:24

xrmx reviewed Apr 28, 2025

View reviewed changes

SrdjanLL reviewed Apr 28, 2025

View reviewed changes

github-actions Bot added the Stale label Mar 11, 2026

github-actions Bot closed this Mar 25, 2026

github-project-automation Bot moved this from Reviewed PRs that need fixes to Done in Python PR digest Mar 25, 2026

Conversation

stanek-michal commented Apr 3, 2025

Description

Type of change

How Has This Been Tested?

Does This PR Require a Core Repo Change?

Checklist:

Uh oh!

linux-foundation-easycla Bot commented Apr 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xrmx left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xrmx commented Apr 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Mar 11, 2026

Uh oh!

github-actions Bot commented Mar 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

linux-foundation-easycla Bot commented Apr 3, 2025 •

edited

Loading

xrmx left a comment •

edited

Loading