v89

Read about the Next Generation Datadog Lambda Extension.

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:89
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:89
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:89
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:89

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:89
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:89
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:89
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:89

What's Changed

Features

• DogStatsD Improvements

  • Added support for DD_STATSD_METRIC_NAMESPACE environment variable to prefix custom metrics with a namespace (#920, #921 by @jchrostek-dd and @litianningdatadog)

• SnapStart Support

  • Added automatic creation of SnapStart spans and metrics when Lambda SnapStart is enabled (#911 by @jchrostek-dd)
  • Replaces init spans with restore spans for SnapStart invocations

• Trace Agent Improvements

  • Increased trace payload limit from 2 MiB to 50 MiB to prevent connection errors with large trace payloads (#902 by @lym953)
  • Implemented libdatadog's native concentrator for trace stats aggregation (#887 by @lym953)
  • Removed hostname from ClientStatsPayload for proper stats aggregation (#905 by @lym953)

• Secrets Management

• SSM Parameter Store: Add support for retrieving Datadog API keys from AWS Systems Manager Parameter Store via the DD_API_KEY_SSM_ARN environment variable. Supports both String and SecureString parameter types. (#924 by @jchrostek-dd)

• Tag Handling

  • Updated DD_TAGS parsing to support both comma (,) and space ( ) delimiters, improving compatibility with container images (#916 by @jchrostek-dd)

Bug Fixes

• Logging

  • Fixed log processing rules incorrectly filtering orphan logs (extension logs and error logs) when exclude patterns were configured (#912 by @lym953)
  • Add URL information to proxy flusher error logs for easier identification of failing endpoints (#933 by @litianningdatadog)

• Metrics

  • Fix an issue where negative CPU utilization values could be reported due to timing differences between /proc/uptime and /proc/stat (#929 by @shreyamalpani and @litianningdatadog)
  • Fix an issue where the tmp_free metric was missing after the enhanced metrics refactor. (#937 by @shreyamalpani)

Performance

• Removed invocation lock and replaced with aggregator service pattern (#894 by @jchrostek-dd )
• Refactored enhanced metrics monitoring to use long-running task instead of spawning new tasks per invocation (#901 by @shreyamalpani)
• Migrated trace aggregation to channel-based approach to avoid locking (#909 by @shreyamalpani)

Error Handling

• Improved trace agent error messages with additional context (#900 by @lym953)

Known Issues

• Trace stats may show up to 9% under/over-counting in high-volume, high-concurrency scenarios (under investigation)

Full Changelog: v88...v89

v88

Read about the Next Generation Datadog Lambda Extension.

Caution

This v88 release no longer contains the Compatibility Mode (Datadog Go Agent).

DD_EXTENSION_VERSION=compatibility will no longer work to fallback. Yet you should not have any problems, this will not disturb your Lambdas or their execution.

Warning

.NET 6 users with DD_SERVERLESS_APPSEC_ENABLED should avoid upgrading, as an issue with the AWS Lambda Runtime conflicts with the HTTP2 header standard. An alternative is upgrading to the .NET 8 runtime.

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:88
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:88
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:88
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:88

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:88
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:88
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:88
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:88

What's Changed

• strip symbols too not just debug info by @astuyve in #876
• fix: dupe copy of saluki fixed by @astuyve in #877
• chore: [Trace Stats] Rename env var DD_COMPUTE_TRACE_STATS by @lym953 in #875
• fix: setting last_flush to 0 makes us flush on the first invocation by @astuyve in #843
• chore(extension): allow register with name by @duncanista in #881
• chore: Add a space in debug log for duration by @lym953 in #880
• chore(gitlab): remove Go agent from build by @duncanista in #795
• fix(logs): use channel-based aggregator by @shreyamalpani in #879
• feat: remove failover to go by @astuyve in #882
• chore: add prefix to some logs by @duncanista in #884
• chore: cleanup code for main agent creation by @duncanista in #885
• fix: tombstone event to be sent properly by @duncanista in #886
• feat: Aggregate tasks into chunks sooner, then drop the lock by @astuyve in #889
• allow native certs by @astuyve in #892
• fix: use None if propagation style is invalid by @astuyve in #895
• feat: Support periodic reload for api key secret by @lym953 in #893

Full Changelog: v87...v88

v87

Read about the Next Generation Datadog Lambda Extension.

Caution

Next release v88 will no longer contain the Datadog Lambda Extension (Datadog Go Agent) in Compatibility Mode.

DD_EXTENSION_VERSION=compatibility will no longer work on the next release. Yet you should not have any problems, this will not disturb your Lambdas or their execution.

This v87 release still contains the Datadog Lambda Extension (Datadog Go Agent) in Compatibility Mode.

Important

Compatibility (previous) Extension Agent version is v7.66.1

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:87
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:87
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:87
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:87

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:87
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:87
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:87
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:87

What's Changed

• fix: remove blocking catchup flush by @astuyve in #821
• feat: eat: Add hierarchical configurable compression levels by @litianningdatadog in #800
• chore: unused deps by @yoshi-taka in #804
• fix(logs): remove sync mutex from logs by @duncanista in #832
• chore(aap): use tokio::sync::Mutex instead of std::sync::Mutex by @RomainMuller in #835
• feat(aap): report _dd.runtime_family when AAP is enabled by @RomainMuller in #836
• chore(extension): moves code outside of binary code by @duncanista in #830
• Support OpenTelemetry Trace Operation Name V1 by @shreyamalpani in #838
• feat: Do not trigger the race flush if an ongoing continuous flush is… by @astuyve in #828
• feat: [Trace Stats] Add feature flag DD_COMPUTE_TRACE_STATS by @lym953 in #841
• fix: Skip log message processing when logs are disabled by @litianningdatadog in #831
• fix: OTLP Database Span Naming by @shreyamalpani in #847
• fix: use tokio time instead of std time because tokio time can be frozen by @astuyve in #846
• chore(extension): move telemetry to extension as module by @duncanista in #848
• feat: don't spawn tmp/proc tasks if one is still running by @astuyve in #851
• fix(aap): lower severity of several warnings by @RomainMuller in #852
• fix: Fix deadlock when snap start and using secret for api key by @lym953 in #853
• fix: remove extra task by @astuyve in #854
• feat: [Trace Stats] Add skeleton of concentrator by @lym953 in #842
• Refactor flush decision logic from if-else chain to match expression pattern by @litianningdatadog in #829
• feat: [Trace Stats] Move stats generation after trace obfuscation by @lym953 in #855
• add support for observability pipeline by @jchrostek-dd in #826
• chore: [Trace Stats] Rename: SendingTraceStatsProcessor -> StatsGenerator by @lym953 in #857
• chore: [Trace Stats] Rename: SendingTraceStatsProcessor -> StatsGener… by @lym953 in #861
• fix: use joinset for trace flushing by @astuyve in #865
• fix: strip debug symbols by @astuyve in #866
• feat: lower zstd default compression by @astuyve in #867
• fix(bottlecap): run-time dependency on libgcc_s.so by @RomainMuller in #868
• chore(bottlecap): stop using the dynamic feature of libddwaf by @RomainMuller in #869
• bump by @astuyve in #870
• feat: [Trace Stats] Implement stats concentrator by @lym953 in #856
• feat: [Trace Stats] Handle tracer lang, tracer version and runtime_id by @lym953 in #863
• feat: [Trace Stats] Handle hostname and env by @lym953 in #862
• fix: Drop guard so we don't hold the lock when we may get traces at a… by @astuyve in #873
• feat: [Trace Stats] Handle container_id and is_trace_root by @lym953 in #872
• feat: v87 by @astuyve in #874

New Contributors

• @yoshi-taka made their first contribution in #804

Full Changelog: v86...v87

v86

Read about the Next Generation Datadog Lambda Extension.

Important

Compatibility (previous) Extension Agent version is v7.66.1

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:86
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:86
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:86
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:86

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:86
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:86
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:86
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:86

What's Changed

• Aj/dsd channel aggregator by @astuyve in #801
• fix: use a joinset for trace flushing by @astuyve in #793
• Ignoring Unwanted Resources in APM by @jchrostek-dd in #794
• fix(lifecycle): spawn tasks on listener by @duncanista in #796

Full Changelog: v85...v86

v85

Read about the Next Generation Datadog Lambda Extension.

Warning

Within the first hours of the release, we found an issue for container images, in which the binaries where wrongly saved outside the /opt folder. This has been resolved by pushing into the same latest image tags.
If you're encountering an issue, remove the pulled image from the cache, and pull it again. If you are having any issues, feel free to contact us by opening an issue in the repo, or through email in [email protected], or by filing a ticket here.

Important

Compatibility (previous) Extension Agent version is v7.66.1
This v85 release should cover feature parity with Go Agent.

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:85
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:85
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:85
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:85

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:85
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:85
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:85
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:85

What's Changed

• fix: Increment OOM metric for provided.al runtimes by @lym953 in #779
• fix: Upgrade libdatadog to mute an error by @lym953 in #781
• vulnerability: upgrade go to 1.24.6 by @duncanista in #782
• chore(logs): No retries while running into permission issue when flushing logs. by @litianningdatadog in #784
• Clear aggregator queue when API key resolution fails to avoid retries… by @litianningdatadog in #785
• fix(traces): Drop aggregated trace data on API key failure to reduce memory usage by @litianningdatadog in #787
• chore: Update error message when fail to send log by @lym953 in #790
• feat: Early load ustr by @lym953 in #792
• chore: lifecycle listener to axum by @astuyve in #789
• remove unused event bus by @astuyve in #797
• feat: port of Serverless AAP from Go to Rust by @RomainMuller in #755

New Contributors

• @litianningdatadog made their first contribution in #784
• @joeyzhao2018 made their first contribution in #733

Full Changelog: v84...v85

v84

Read about the Next Generation Datadog Lambda Extension.

Warning

This release contains some breaking changes (#751)

• Service names are now reflect the actual AWS resource name instead of being vague. The old to new mappings are:
  • aws.lambda → [function_name]
  • aws.dynamodb → [table_name]
  • aws.sns → [topic_name]
  • aws.sqs → [queue_name]
  • aws.kinesis → [stream_name]
  • aws.s3 → [bucket_name]
  • aws.eventbridge → [event_name]
• To opt out and restore old behavior, use the env var:

  DD_TRACE_AWS_SERVICE_REPRESENTATION_ENABLED=false

• Dashboards, monitors, or alerts using service as a filter may break.
  • Recommended fix: Switch to using operation_name with same value to achieve old behavior.
• Sampling rules may need to be updated to reflect the new service names

Important

Compatibility (previous) Extension Agent version is v7.66.1

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:84
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:84
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:84
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:84

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:84
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:84
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:84
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:84

What's Changed

• feat(apm): aggregate and flush data accordingly for DSM, Profiling, LLMObs, and Live Debugger by @duncanista in #737
• chore: pass SendDataBuilderInfo instead of SendData until flush time by @lym953 in #745
• fix(fips): ignore test and build deps by @duncanista in #750
• feat: Lazily resolve api key by @lym953 in #717
• feat: Handle API key resolution failure by @lym953 in #732
• feat: dual shipping APM support by @shreyamalpani in #735
• chore: Add doc and rename function for flushing strategy by @lym953 in #740
• chore(cargo): use dd-rust-license-tool instead of bundle-licenses by @duncanista in #752
• build(deps): bump anchore/scan-action from 6.4.0 to 6.5.0 by @dependabot[bot] in #753
• chore: upgrade to edition 2024 and fix all linter warnings by @RomainMuller in #754
• chore(dependencies): remove datadog-trace-agent by @duncanista in #757
• chore(event-bus): clean and remove unused code by @duncanista in #758
• chore: Change trace payload size limit from 3.2 MiB to 3.2 MB by @lym953 in #763
• chore: Add/update doc for aggregators by @lym953 in #764
• feat: Bump libdatadog by @astuyve in #769
• fix(proxy): ensure correct headers are set when forwarding request by @duncanista in #768
• build(deps): bump anchore/scan-action from 6.5.0 to 6.5.1 by @dependabot[bot] in #774
• feat: add metric to track proactive initialization with no invocations by @jchrostek-dd in #775
• feat: For OOM metric, also use error type in PlatformRuntimeDone by @lym953 in #776
• fix(proxy): stream requests by @duncanista in #771
• fix: Fix parsing provided.al runtime by reading /etc/os-release by @lym953 in #778
• feat: Support proxying instrumentation telemetry API payloads by @jchrostek-dd in #777
• fix(apm): Enhance Synthetic Span Service Representation by @zarirhamza in #751

New Contributors

• @jchrostek-dd made their first contribution in #775
• @zarirhamza made their first contribution in #751

Full Changelog: v83...v84

v83

Read about the Next Generation Datadog Lambda Extension.

Important

Compatibility (previous) Extension Agent version is v7.66.1

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:83
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:83
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:83
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:83

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:83
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:83
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:83
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:83

What's Changed

• chore(otlp): move agent to axum by @duncanista in #711
• chore: bump go to 1.24.4 by @apiarian-datadog in #720
• chore: Separate AwsCredentials from AwsConfig by @lym953 in #716
• chore(config): separate config from sources by @duncanista in #709
• fix: move env mapping from ignore_missing_datadog_fields=true to false block by @rcdailey in #724
• Dual Shipping Logs Support by @shreyamalpani in #718
• Aj/migrate trace agent to axum by @astuyve in #719
• chore(trace-agent): clean state handling by @duncanista in #730
• chore(telemetry-api): move listener to axum by @duncanista in #729
• feat: Add jemalloc by @astuyve in #736
• chore: Upgrade libdatadog by @lym953 in #744
• Aj/fix shutdown deadlock by @astuyve in #746

New Contributors

• @rcdailey made their first contribution in #724

Full Changelog: v82...v83

v82

Read about the Next Generation Datadog Lambda Extension.

Important

Compatibility (previous) Extension Agent version is v7.66.1

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:82
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:82
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:82
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:82

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:82
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:82
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:82
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:82

What's Changed

• feat: set init_type on init duration metric by @astuyve in #700
• feat: Allow users to specify continuous strategy by @astuyve in #701
• feat: Use http2 unless overridden or using a proxy by @astuyve in #706
• Dual shipping metrics support by @shreyamalpani in #704
• chore: Fix typos in README by @lym953 in #703
• feat: use skip missed tick behavior by @astuyve in #708
• feat: parallelize batch flusher by @astuyve in #707
• v82 by @astuyve in #710

Full Changelog: v81...v82

v81

Read about the Next Generation Datadog Lambda Extension.

Important

Compatibility (previous) Extension Agent version is v7.66.1

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:81
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:81
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:81
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:81

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:81
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:81
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:81
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:81

What's Changed

• feat: continuous flushing strategy for high throughput functions by @astuyve in #684
• Combine handle_next_invocation and next_event. Don't block for PlatformReport on timeout shutdowns by @astuyve in #692
• fix: deadlock in main loop by @astuyve in #696
• fix: bump flush_timeout default by @astuyve in #697

Compatibility Extension

• Fixes an issue where ASM fails to start in DataDog/datadog-agent#37639

Full Changelog: v80...v81

v80

Read about the Next Generation Datadog Lambda Extension.

Warning

There is a critical issue with ASM/AppSec and v80. ASM users should upgrade to v81 or disable ASM (remove DD_SERVERLESS_APPSEC_ENABLED).

Important

Compatibility (previous) Extension Agent version is v7.66.0

# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:80
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:80
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:80
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:80

# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:80
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:80
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:80
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:80

What's Changed

• chore(gitlab): update expire time for artifacts to 1 weeks by @duncanista in #689
• v80 by @astuyve in #690

Compatibility Extension

• Fixes an issue where compression errors are logged erroneously in DataDog/datadog-agent#37523

Full Changelog: v79...v80