feat: API Gateway Tracing #3095

PROFeNoM · 2025-02-19T12:13:14Z

Description

Telemetry PR: https://github.com/DataDog/dd-go/pull/171137

Reviewer checklist

Test coverage seems ok.
Appropriate labels assigned.

…happens

codecov-commenter · 2025-02-19T12:19:20Z

Codecov Report

Attention: Patch coverage is 49.01961% with 26 lines in your changes missing coverage. Please review.

Project coverage is 72.87%. Comparing base (445f72a) to head (aaac753).
Report is 7 commits behind head on master.

Files with missing lines	Patch %	Lines
.../DDTrace/Integrations/Swoole/SwooleIntegration.php	10.52%	17 Missing ⚠️
.../Integrations/Roadrunner/RoadrunnerIntegration.php	71.87%	9 Missing ⚠️

❌ Your patch status has failed because the patch coverage (49.01%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3095      +/-   ##
============================================
- Coverage     74.74%   72.87%   -1.88%     
- Complexity     2791     2806      +15     
============================================
  Files           112      139      +27     
  Lines         11044    15317    +4273     
  Branches          0     1047    +1047     
============================================
+ Hits           8255    11162    +2907     
- Misses         2789     3602     +813     
- Partials          0      553     +553

Flag	Coverage Δ
appsec-extension	`68.28% <ø> (?)`
tracer-php	`74.62% <49.01%> (-0.12%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
.../Integrations/Roadrunner/RoadrunnerIntegration.php	`70.71% <71.87%> (-1.96%)`	⬇️
.../DDTrace/Integrations/Swoole/SwooleIntegration.php	`43.44% <10.52%> (-3.48%)`	⬇️

... and 27 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 445f72a...aaac753. Read the comment docs.

pr-commenter · 2025-02-19T12:42:50Z

Benchmarks [ tracer ]

Benchmark execution time: 2025-02-20 10:25:11

Comparing candidate commit 6aecc1f in PR branch alex/AIDM-548_api-gateway-bis with baseline commit 445f72a in branch master.

Found 2 performance improvements and 3 performance regressions! Performance is the same for 173 metrics, 0 unstable metrics.

scenario:ContextPropagationBench/benchInject128Bit-opcache

🟥 execution_time [+48.083ns; +121.917ns] or [+2.890%; +7.327%]

scenario:EmptyFileBench/benchEmptyFileBaseline

🟥 execution_time [+67.659µs; +197.281µs] or [+2.279%; +6.645%]

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

🟥 execution_time [+6.197µs; +8.203µs] or [+2.885%; +3.819%]

scenario:PDOBench/benchPDOBaseline

🟩 execution_time [-17.022µs; -15.824µs] or [-8.451%; -7.856%]

scenario:SamplingRuleMatchingBench/benchRegexMatching3-opcache

🟩 execution_time [-459.266ns; -165.534ns] or [-6.261%; -2.257%]

wantsui · 2025-02-19T15:49:41Z

tests/Integrations/Laravel/APIGWTest.php

+            'DD_ENV' => 'local-test',
+            'DD_VERSION' => '1.0',
+            'DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED' => 'true',
+            //'DD_TRACE_HEADER_TAGS' => 'x-dd-proxy-domain-name,x-dd-proxy,x-dd-proxy-httpmethod,x-dd-proxy-path,x-dd-proxy-request-time-ms,x-dd-proxy-stage',


Should this be commented out?

tests/Integrations/Laravel/APIGWTest.php

wantsui · 2025-02-19T15:56:18Z

tests/snapshots/tests.integrations.laravel.apigw_test.test_inferred_proxy.json

@@ -0,0 +1,381 @@
+[[
+  {
+    "name": "aws-apigateway",


The span name in the Python, Go, and JS implementation is aws.apigateway. Any way to change this?

zarirhamza · 2025-02-20T15:38:01Z

tests/Integrations/Laravel/APIGWTest.php

+                    'A simple GET request returning a string',
+                    '/simple?key=value&pwd=should_redact',
+                    [
+                        'x-dd-proxy: aws.apigateway',


The way we implemented it in other languages was to have a map w/the header value of aws-apigateway to map to a span name value of aws.apigateway. We want the header value to remain unchanged and we want the span name to be slightly different which can be represented by a key value map.

That way we can add more values in the future if we have more proxy headers that correspond to different operation names

Refer to this as an example

Note, they also include the component value in the map which would change for future proxies as well that may not necessarily be equal to the proxy header

bwoebi · 2025-02-21T10:28:49Z

ext/ddtrace.c

    UNUSED(old_value, new_str);

    ddtrace_span_properties *pspan = ddtrace_active_span_props();
    while (pspan) {
+        if (skip_inferred) {
+            ddtrace_span_data *span = SPANDATA(pspan);
+            if (span && span->type == DDTRACE_INFERRED_SPAN) {


Suggested change

if (span && span->type == DDTRACE_INFERRED_SPAN) {

if (span->type == DDTRACE_INFERRED_SPAN) {

Nit, but SPANDATA() is just an offset calculation from pspan and will never be NULL.

bwoebi · 2025-02-21T10:34:52Z

ext/inferred_proxy_headers.c

+    ddtrace_proxy_info *info;
+    ZEND_HASH_FOREACH_PTR(&proxy_info_map, info) {
+        efree(info);
+    } ZEND_HASH_FOREACH_END();


Nit: the idiomatic way would be using:

static dd_proxy_info_dtor(zval *zv) { efree(Z_PTR_P(zv)); }

and passing it to zend_hash_init as destructor.

bwoebi · 2025-02-21T10:35:40Z

ext/inferred_proxy_headers.c

+}
+
+void ddtrace_init_proxy_info_map(void) {
+    zend_hash_init(&proxy_info_map, 8, NULL, NULL, 1);


This should be 0 as last parameter right? persistent=1 is for memory supposed to survive the request cycle.

Oops, indeed. I initially wanted to do the initialization/free in MINIT/MSHUTDOWN, but faced issues. Nice catch, ty.

this should be in minit/mshutdown though. No point in reinitializing that every request.

bwoebi · 2025-02-21T12:32:02Z

ext/serializer.c

+    } else if (span->std.ce == ddtrace_ce_root_span_data) {
+        ddtrace_root_span_data *root = ROOTSPANDATA(&span->std);
+        if (root->child_root) { // Can't check for DDTRACE_INFERRED_SPAN because the span is now closed
+            zval *child_exception_zv = &root->child_root->span.property_exception;
+            has_exception = Z_TYPE_P(child_exception_zv) == IS_OBJECT && instanceof_function(Z_OBJCE_P(child_exception_zv), zend_ce_throwable);


Can you move this before the if (has_exception) above?

// inherit exception on inferred span from child root if (!has_exception && span->std.ce == ddtrace_ce_root_span_data) { ddtrace_root_span_data *root = ROOTSPANDATA(&span->std); if (root->child_root) { // Can't check for DDTRACE_INFERRED_SPAN because the span is now closed exception_zv = &root->child_root->span.property_exception; has_exception = Z_TYPE_P(child_exception_zv) == IS_OBJECT && instanceof_function(Z_OBJCE_P(child_exception_zv), zend_ce_throwable); } }

The duplication was confusing me

bwoebi · 2025-02-21T12:38:20Z

ext/span.h

@@ -86,6 +87,7 @@ struct ddtrace_span_data {
    bool notify_user_req_end;
    struct ddtrace_span_data *next;
    struct ddtrace_root_span_data *root;
+    bool is_child_of_inferred_span;


Why do we have this property at all? Isn't it redundant with child_root != NULL on the root_span_data?

I see only one usage of it, and it's trivially replaced.

bwoebi · 2025-02-21T12:45:12Z

ext/span.h

@@ -107,6 +109,7 @@ struct ddtrace_root_span_data {
    ddtrace_rule_result sampling_rule;
    bool explicit_sampling_priority;
    enum ddtrace_trace_limited trace_is_limited;
+    struct ddtrace_root_span_data *child_root; // Only used when inferring proxy services (type: DDTRACE_INFERRED_SPAN)


let's rename it to inferred_root

PROFeNoM added 30 commits February 7, 2025 11:23

Base Implementation

f0c4bf2

fix: segfault

4d706fb

fix: duration

39f9f2a

fix: duration

fccd91b

error log start/duration/duration_start

2bbf531

fix: duration

291f75b

x-dd-proxy-domain-name add debug logs

6fc1632

service debug logs

162b382

tests: Laravel x InferredProxy

641811a

fix: Specifically do not inherit service property

144de18

more debug logs

ba71c60

fix: Don't overwrite inferred span's service.

9fb4ab9

style: refactor service alter skip

7153541

feat: Octane Swoole support

03c0569

style: Refactor using DDTRACE_INFERRED_SPAN type

cf65501

tests: Add some (unfinished) tests

a1c525a

feat: Very quick Roadrunner support (to be enhanced)

bd91a71

Set _dd.inferred_span to 1

bcc25af

tests: Clean EXPECTF section

d4b2745

fix: Memory leak

e91cc69

chore: update snapshots

dbd2e66

tests: Distributed Tracing

3e1945c

tests: Exceptions in Roadrunner/Swoole

969ff8b

fix: Laravel Latest CLI Test Script Location

18a1527

fix: persisting DD_SERVICE on Laravel Tests

8c22cd5

fix: Roadrunner tests order

42e3581

fix: Laravel Connection issue

cf59fa3

fix: Laravel Connection issue

509bc51

error output

4c29c16

error output

340b32e

PROFeNoM added 5 commits February 18, 2025 16:21

Try to disable DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED and see what …

5fb7511

…happens

Try to split the tests

572fbfd

Remove logs

f54687a

Totally split the tests

ec98ccd

style: Use ddtrace_read_header

ed31db3

PROFeNoM self-assigned this Feb 19, 2025

style: Remove some debug logs

f0ccfa3

style: Remove some debug logs

43f3bf1

PROFeNoM changed the title ~~wip~~ feat: API Gateway Tracing Feb 19, 2025

Merge branch 'refs/heads/master' into alex/AIDM-548_api-gateway-bis

e0bbd7a

wantsui requested changes Feb 19, 2025

View reviewed changes

PROFeNoM added 5 commits February 20, 2025 10:07

style: Change x-dd-proxy value from aws-apigateway to aws.apigateway

d39e0b7

style: Remove duration from serialization debug logs

6aecc1f

fix: Prevent segfaults/mem leak on missing headers

926dd2b

style: one-liner

bdb31b5

style: change test description

e5707d7

zarirhamza requested changes Feb 20, 2025

View reviewed changes

Add a hashmap of supported proxies

aaac753

bwoebi reviewed Feb 21, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: API Gateway Tracing #3095

feat: API Gateway Tracing #3095

PROFeNoM commented Feb 19, 2025 •

edited

Loading

codecov-commenter commented Feb 19, 2025 •

edited

Loading

pr-commenter bot commented Feb 19, 2025 •

edited

Loading

wantsui Feb 19, 2025

wantsui Feb 19, 2025

zarirhamza Feb 20, 2025

zarirhamza Feb 20, 2025 •

edited

Loading

bwoebi Feb 21, 2025

bwoebi Feb 21, 2025

bwoebi Feb 21, 2025

PROFeNoM Feb 21, 2025

bwoebi Feb 21, 2025

bwoebi Feb 21, 2025 •

edited

Loading

bwoebi Feb 21, 2025

bwoebi Feb 21, 2025

bwoebi Feb 21, 2025

	if (span && span->type == DDTRACE_INFERRED_SPAN) {
	if (span->type == DDTRACE_INFERRED_SPAN) {

feat: API Gateway Tracing #3095

Are you sure you want to change the base?

feat: API Gateway Tracing #3095

Conversation

PROFeNoM commented Feb 19, 2025 • edited Loading

Description

Reviewer checklist

codecov-commenter commented Feb 19, 2025 • edited Loading

Codecov Report

pr-commenter bot commented Feb 19, 2025 • edited Loading

Benchmarks [ tracer ]

scenario:ContextPropagationBench/benchInject128Bit-opcache

scenario:EmptyFileBench/benchEmptyFileBaseline

scenario:MessagePackSerializationBench/benchMessagePackSerialization-opcache

scenario:PDOBench/benchPDOBaseline

scenario:SamplingRuleMatchingBench/benchRegexMatching3-opcache

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zarirhamza Feb 20, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bwoebi Feb 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PROFeNoM commented Feb 19, 2025 •

edited

Loading

codecov-commenter commented Feb 19, 2025 •

edited

Loading

pr-commenter bot commented Feb 19, 2025 •

edited

Loading

zarirhamza Feb 20, 2025 •

edited

Loading

bwoebi Feb 21, 2025 •

edited

Loading