Add OTEL gen-ai message format for auto-instrumented message tracing by fpfp100 · Pull Request #231 · microsoft/Agent365-nodejs

fpfp100 · 2026-04-09T01:36:43Z

Summary

Add auto-instrumented input/output message recording in OTEL gen-ai format (version: "0.1.0") for both OpenAI and LangChain extensions — content attributes are recorded unconditionally (no opt-in gating)
Fix LangChain v1 compatibility: tool results, tool call IDs, system instructions, token usage, model name, and message type detection now handle v1 (LangGraph) formats
Add message-schema-validator test helper and contract test suites for both extensions
Use shared MAX_SPAN_SIZE_BYTES constant for binary field truncation instead of hardcoded values

Note: This PR is built on top of #228 (structured object support for tool call arguments/responses). It should be merged after #228.

Changes

OpenAI Extension

OpenAIAgentsTraceProcessor: Auto-records structured InputMessages/OutputMessages on response, generation, and function spans; all string fallbacks now use versioned envelope
Utils.ts: New buildStructuredInputMessages() and buildStructuredOutputMessages() for OTEL gen-ai schema conversion; removed unused getAttributesFromInput; uses MAX_SPAN_SIZE_BYTES for truncation

LangChain Extension

LangChainTracer: Content attributes are recorded unconditionally — removed isContentRecordingEnabled option
Utils.ts: New structured message builders (setInputMessagesAttribute, setOutputMessagesAttribute); v1 fixes for setToolAttributes, setSystemInstructionsAttribute, setTokenAttributes, getModel, getMessageType

Core

Export new OTEL gen-ai message types and serialization utilities from index.ts
Export MAX_SPAN_SIZE_BYTES from observability package

Test plan

All 1206 unit tests pass (62 suites)
LangChain sample: validated spans with tool call flow (get_weather) — all attributes correct
OpenAI sample: validated spans with MCP tools + LLM response — all attributes correct
Code review findings (CRM-001 through CRM-013) addressed

🤖 Generated with Claude Code

Copilot

Pull request overview

Adds OTEL gen-ai (version: "0.1.0") structured message recording across core scopes and the OpenAI/LangChain observability extensions, with gating via isContentRecordingEnabled and expanded compatibility for LangChain v1/LangGraph shapes.

Changes:

Introduces structured gen_ai.input.messages / gen_ai.output.messages recording (versioned envelope) and exports related message utilities/types from core.
Updates LangChain extension attribute extraction for v1 formats (tool outputs/call IDs, system instructions, token usage, model name, message detection).
Adds a shared message schema validator helper plus new contract tests and updated unit tests for new serialization/overwrite semantics.

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/observability/tracing/message-utils.test.ts	Removes `isStringArray` tests; focuses on wrapper detection/normalization behavior.
tests/observability/extension/openai/OpenAIMessageContract.test.ts	Adds OpenAI OTEL message contract tests for structured input/output builders and edge cases.
tests/observability/extension/openai/OpenAIAgentsTraceProcessor.test.ts	Updates OpenAI trace processor tests to assert versioned message envelopes + schema validation.
tests/observability/extension/langchain/LangChainObservabilityAttributes.test.ts	Updates expected message attributes to versioned envelopes; adds v1 coverage for tool/system/tokens.
tests/observability/extension/langchain/LangChainMessageContract.test.ts	Adds LangChain message contract tests using real LangChain message types + LangGraph path.
tests/observability/extension/hosting/TurnContextUtils.test.ts	Removes execution-type baggage tests consistent with removed API/constant.
tests/observability/extension/hosting/scope-utils.test.ts	Removes tenant-details derivation tests aligned with API removal.
tests/observability/extension/helpers/message-schema-validator.ts	Adds shared test helper for validating serialized OTEL message envelopes/parts.
tests/observability/core/scopes.test.ts	Refactors exporter setup; adds tests for request content → input messages + safe JSON serialization.
tests/observability/core/output-scope.test.ts	Updates OutputScope to overwrite semantics; adds tests for dict outputs and single-string paths.
tests/observability/core/agent365-exporter.test.ts	Adds truncation coverage for raw-dict `gen_ai.output.messages` vs versioned wrappers.
packages/agents-a365-observability/src/tracing/util.ts	Adds `safeSerializeToJson` helper for consistent JSON-safe span attributes.
packages/agents-a365-observability/src/tracing/scopes/OutputScope.ts	Changes output message recording to overwrite semantics; supports raw dict outputs.
packages/agents-a365-observability/src/tracing/scopes/OpenTelemetryScope.ts	Updates docs/behavior to accept single-string message params (input/output).
packages/agents-a365-observability/src/tracing/scopes/InvokeAgentScope.ts	Auto-records `Request.content` as input messages; simplifies response recording API usage.
packages/agents-a365-observability/src/tracing/scopes/ExecuteToolScope.ts	Allows spanKind override; serializes tool args/results via `safeSerializeToJson`.
packages/agents-a365-observability/src/tracing/message-utils.ts	Supports single-string message params; removes `isStringArray` guard.
packages/agents-a365-observability/src/tracing/exporter/utils.ts	Tightens “versioned wrapper” detection; improves shrink-action control flow; adds raw-JSON sentinel behavior.
packages/agents-a365-observability/src/tracing/contracts.ts	Removes `ExecutionType`/`TenantDetails`; widens message/tool argument types; introduces `ResponseMessagesParam`.
packages/agents-a365-observability/src/tracing/constants.ts	Removes `GEN_AI_EXECUTION_TYPE_KEY`.
packages/agents-a365-observability/src/index.ts	Updates public exports (types removed/added; exports message utilities).
packages/agents-a365-observability/docs/design.md	Removes `ExecutionType` enum from design docs.
packages/agents-a365-observability-hosting/src/utils/TurnContextUtils.ts	Removes execution type extraction logic and related imports.
packages/agents-a365-observability-hosting/src/utils/ScopeUtils.ts	Removes tenant-details derivation helper and import.
packages/agents-a365-observability-hosting/src/utils/BaggageBuilderUtils.ts	Removes execution-type baggage population.
packages/agents-a365-observability-hosting/src/middleware/BaggageMiddleware.ts	Removes execution-type baggage injection in middleware.
packages/agents-a365-observability-hosting/docs/design.md	Updates hosting design docs to remove execution-type references.
packages/agents-a365-observability-extensions-openai/src/Utils.ts	Adds OTEL structured message builders/wrappers for OpenAI input/output formats.
packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceProcessor.ts	Auto-records structured input/output messages on response spans; removes execution-type filtering.
packages/agents-a365-observability-extensions-langchain/src/Utils.ts	Adds structured message attribute builders + v1-compatible extraction for tool/system/tokens/model.
.codereviews/claude-pr-toolupdate-20260408_154355.md	Adds consolidated external review artifact documenting prior findings/resolutions.

…tput message tracing - OpenAI extension: Auto-record structured InputMessages/OutputMessages on response, generation, and function spans via OpenAIAgentsTraceProcessor with isContentRecordingEnabled option - LangChain extension: Auto-record structured InputMessages/OutputMessages on chat, invoke_agent, and execute_tool spans via LangChainTracer with isContentRecordingEnabled option - LangChain v1 support: Fix setToolAttributes to handle plain string tool results (v1 format), fix tool_call_id extraction from inputs, fix setSystemInstructionsAttribute to use getMessageType() for all message formats, fix setTokenAttributes to check last message for usage_metadata, fix getModel to check v1 direct response_metadata path - Add message-schema-validator test helper for validating OTEL gen-ai message contract - Add LangChainMessageContract and OpenAIMessageContract test suites Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- CRM-001: Move GEN_AI_TOOL_TYPE_KEY setAttribute outside forEach loop - CRM-002: Wrap string output/input fallbacks in versioned OTEL message envelope - CRM-003: Support tokenUsage shape (promptTokens/completionTokens) in setTokenAttributes - CRM-009: Remove unused getAttributesFromInput export - CRM-010: Replace dead extractMessageContent with extractStringContent delegating to extractRawContent - CRM-011: Fix self-referential comment - CRM-006: Add direct v1 constructor format tests for input/output messages - CRM-007: Add response span content-gating suppression test - CRM-008: Add empty array and null entry edge case tests - CRM-013: Add function_call, input_file, malformed JSON, and unknown block type tests - Fix getMessageType to skip v1 'constructor' type and check id array for message class Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 8 comments.

…tensions, align tool serialization - Remove isContentRecordingEnabled toggle: content recording is now ON by default (aligned with Python/.NET SDKs). suppressInvokeAgentInput remains. - Remove truncateValue from both extensions: truncation now happens at the span/exporter level in core, not pre-truncated in extensions. - Use safeSerializeToJson for tool arguments/results in both extensions, matching the core ExecuteToolScope pattern from PR #228. - Export safeSerializeToJson from core index.ts. - Add tool argument validation tests for both OpenAI and LangChain extensions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

@param

…JSDoc - processResponseSpanData: add else fallback for non-array output via wrapRawContentAsOutputMessages (handles object output) - mapInputContentBlock: map input_image to type 'blob' (matches BlobPart contract) instead of non-standard 'image' - Remove stale @param options JSDoc from LangChainTraceInstrumentor.instrument Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceInstrumentor.ts:1

Removing isContentRecordingEnabled from the public OpenAIAgentsInstrumentationConfig (and related constructor options) is a breaking API change for consumers. Consider keeping the option (even if ignored) and deprecating it first, or bump the major version and clearly document the behavioral change.

// ------------------------------------------------------------------------------

…ility, add missing expect import - Replace hardcoded 1024 with MAX_SPAN_SIZE_BYTES in OpenAI extensions Utils - Export MAX_SPAN_SIZE_BYTES from observability package - Add missing `import { expect } from '@jest/globals'` to test helper Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

packages/agents-a365-observability-extensions-openai/src/OpenAIAgentsTraceInstrumentor.ts:1

The public OpenAIAgentsInstrumentationConfig option isContentRecordingEnabled was removed, which is a breaking change for downstream consumers (and also changes behavior since content is now recorded unconditionally elsewhere in this PR). To avoid an unexpected breaking change, either (mandatory) reintroduce the option with the previous default (false) and keep the gating behavior, or (alternative) treat this as a major version change and document the behavioral/API change in release notes and package versioning.

// ------------------------------------------------------------------------------

nikhilNava

You can consider adding integration tests will get the exact structure from of input and output messages for auto instrumentation.
This might not cover scenarios like audio/file part etc

Copilot AI review requested due to automatic review settings April 9, 2026 01:36

fpfp100 requested review from a team as code owners April 9, 2026 01:36

Copilot started reviewing on behalf of fpfp100 April 9, 2026 01:37 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

jsl517 and others added 2 commits April 9, 2026 11:54

fpfp100 force-pushed the users/pefan/autoinstrumentmessage branch from 928a65b to 4115a64 Compare April 9, 2026 18:58

Copilot AI review requested due to automatic review settings April 9, 2026 19:54

Copilot started reviewing on behalf of fpfp100 April 9, 2026 19:55 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

fpfp100 force-pushed the users/pefan/autoinstrumentmessage branch from d66e8d4 to 45dd931 Compare April 9, 2026 20:30

fpfp100 force-pushed the users/pefan/autoinstrumentmessage branch from 45dd931 to 22c0127 Compare April 9, 2026 20:37

Copilot AI review requested due to automatic review settings April 9, 2026 20:37

Copilot started reviewing on behalf of fpfp100 April 9, 2026 20:37 View session

Copilot AI reviewed Apr 9, 2026

View reviewed changes

nikhilNava previously approved these changes Apr 10, 2026

View reviewed changes

Comment thread .codereviews/claude-pr-toolupdate-20260408_154355.md Outdated

juliomenendez previously approved these changes Apr 10, 2026

View reviewed changes

fpfp100 dismissed stale reviews from juliomenendez and nikhilNava via a5a8506 April 10, 2026 21:38

Copilot AI review requested due to automatic review settings April 13, 2026 17:24

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Copilot started reviewing on behalf of fpfp100 April 13, 2026 17:38 View session

Copilot AI review requested due to automatic review settings April 13, 2026 18:37

fpfp100 force-pushed the users/pefan/autoinstrumentmessage branch from 7d13038 to 15fc5b7 Compare April 13, 2026 18:37

Copilot AI reviewed Apr 13, 2026

View reviewed changes

Comment thread packages/agents-a365-observability-extensions-langchain/src/tracer.ts

Comment thread tests/observability/extension/helpers/message-schema-validator.ts

Comment thread packages/agents-a365-observability-extensions-langchain/src/tracer.ts

Copilot started reviewing on behalf of fpfp100 April 13, 2026 18:55 View session

nikhilNava approved these changes Apr 13, 2026

View reviewed changes

juliomenendez approved these changes Apr 13, 2026

View reviewed changes

fpfp100 merged commit c493d82 into main Apr 13, 2026
11 checks passed

fpfp100 deleted the users/pefan/autoinstrumentmessage branch April 13, 2026 20:43

Conversation

fpfp100 commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

OpenAI Extension

LangChain Extension

Core

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nikhilNava left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

fpfp100 commented Apr 9, 2026 •

edited

Loading