Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

writer impl #8372

Draft
wants to merge 4 commits into
base: gary/impl-llm-obs-api
Choose a base branch
from
Draft

Conversation

gary-huang
Copy link
Contributor

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Comment on lines 158 to 325
private PayloadV2() {
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Consider adding super() or this() to your constructor (...read more)

In Java, it is suggested to call super() in an extended class. This rule will report a violation if both a call to super() and an overloaded constructor are absent.

View in Datadog  Leave us feedback  Documentation

LOGGER.error("Exception occurred", e);
}
} catch (Exception e) {
e.printStackTrace();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Quality Violation

Avoid printStackTrace(); use a logger call instead. (...read more)

Use a logging framework instead of printStackTrace() when handling exceptions. printStackTrace() can be useful during development for quick debugging, but it is not suitable for production code.

View in Datadog  Leave us feedback  Documentation

@gary-huang gary-huang force-pushed the gary/impl-llm-obs-api branch 2 times, most recently from 680ec94 to b1d7aeb Compare February 13, 2025 07:39
@gary-huang gary-huang force-pushed the gary/add-llm-obs-writer branch from 113bf8c to 9052dc4 Compare February 13, 2025 07:41
@pr-commenter
Copy link

pr-commenter bot commented Feb 13, 2025

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master gary/add-llm-obs-writer
git_commit_date 1740058310 1740066638
git_commit_sha 10dc2ef 9ee77d9
release_version 1.47.0-SNAPSHOT~10dc2ef4fb 1.46.0-SNAPSHOT~9ee77d9810
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1740072326 1740072326
ci_job_id 816161625 816161625
ci_pipeline_id 56442606 56442606
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-y3jyzt9j-project-304-concurrent-2-xec29rxl 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-y3jyzt9j-project-304-concurrent-2-xec29rxl 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 4 performance improvements and 11 performance regressions! Performance is the same for 43 metrics, 5 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast:GlobalTracer worse
[+20.898ms; +26.495ms] or [+9.044%; +11.466%]
254.776ms 231.080ms
scenario:startup:insecure-bank:iast:IAST better
[-1.703ms; -0.657ms] or [-7.427%; -2.865%]
21.745ms 22.924ms
scenario:startup:insecure-bank:iast:Remote Config worse
[+30.805µs; +69.733µs] or [+4.982%; +11.278%]
668.601µs 618.333µs
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:GlobalTracer worse
[+21.206ms; +23.284ms] or [+9.196%; +10.097%]
252.848ms 230.604ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:IAST better
[-1.772ms; -1.049ms] or [-7.704%; -4.562%]
21.584ms 22.995ms
scenario:startup:insecure-bank:iast_HARDCODED_SECRET_DISABLED:Remote Config worse
[+45.579µs; +77.095µs] or [+7.499%; +12.684%]
669.129µs 607.792µs
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:GlobalTracer worse
[+20.106ms; +22.010ms] or [+8.700%; +9.523%]
252.174ms 231.116ms
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Remote Config worse
[+21.701µs; +56.087µs] or [+3.481%; +8.997%]
662.282µs 623.388µs
scenario:startup:insecure-bank:tracing:GlobalTracer worse
[+16.520ms; +25.647ms] or [+6.839%; +10.617%]
262.650ms 241.566ms
scenario:startup:petclinic:appsec:GlobalTracer worse
[+19.659ms; +24.475ms] or [+8.286%; +10.316%]
259.322ms 237.255ms
scenario:startup:petclinic:appsec:AppSec better
[-11.533ms; -9.891ms] or [-6.498%; -5.572%]
166.783ms 177.495ms
scenario:startup:petclinic:iast:GlobalTracer worse
[+16.769ms; +23.214ms] or [+7.206%; +9.975%]
252.714ms 232.723ms
scenario:startup:petclinic:iast:IAST better
[-2.130ms; -0.920ms] or [-9.188%; -3.967%]
21.658ms 23.183ms
scenario:startup:petclinic:profiling:GlobalTracer worse
[+22.364ms; +26.694ms] or [+6.394%; +7.632%]
374.284ms 349.755ms
scenario:startup:petclinic:tracing:GlobalTracer worse
[+17.734ms; +23.513ms] or [+7.361%; +9.760%]
261.536ms 240.913ms
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.05 s) : 0, 1049969
Total [baseline] (8.678 s) : 0, 8677878
Agent [candidate] (1.067 s) : 0, 1067268
Total [candidate] (8.652 s) : 0, 8652270
section iast
Agent [baseline] (1.175 s) : 0, 1174956
Total [baseline] (9.224 s) : 0, 9223609
Agent [candidate] (1.194 s) : 0, 1194029
Total [candidate] (9.285 s) : 0, 9285252
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.173 s) : 0, 1173059
Total [baseline] (9.22 s) : 0, 9220376
Agent [candidate] (1.19 s) : 0, 1189535
Total [candidate] (9.235 s) : 0, 9235294
section iast_TELEMETRY_OFF
Agent [baseline] (1.172 s) : 0, 1172116
Total [baseline] (9.2 s) : 0, 9200386
Agent [candidate] (1.184 s) : 0, 1184350
Total [candidate] (9.246 s) : 0, 9245670
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.05 s -
Agent iast 1.175 s 124.987 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.173 s 123.09 ms (11.7%)
Agent iast_TELEMETRY_OFF 1.172 s 122.147 ms (11.6%)
Total tracing 8.678 s -
Total iast 9.224 s 545.731 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.22 s 542.497 ms (6.3%)
Total iast_TELEMETRY_OFF 9.2 s 522.508 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent iast 1.194 s 126.761 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.19 s 122.267 ms (11.5%)
Agent iast_TELEMETRY_OFF 1.184 s 117.082 ms (11.0%)
Total tracing 8.652 s -
Total iast 9.285 s 632.982 ms (7.3%)
Total iast_HARDCODED_SECRET_DISABLED 9.235 s 583.023 ms (6.7%)
Total iast_TELEMETRY_OFF 9.246 s 593.4 ms (6.9%)
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.232 ms) : 0, 722232
BytebuddyAgent [candidate] (718.601 ms) : 0, 718601
GlobalTracer [baseline] (241.566 ms) : 0, 241566
GlobalTracer [candidate] (262.65 ms) : 0, 262650
AppSec [baseline] (55.78 ms) : 0, 55780
AppSec [candidate] (57.929 ms) : 0, 57929
Remote Config [baseline] (705.115 µs) : 0, 705
Remote Config [candidate] (712.939 µs) : 0, 713
Telemetry [baseline] (14.433 ms) : 0, 14433
Telemetry [candidate] (12.338 ms) : 0, 12338
section iast
BytebuddyAgent [baseline] (838.655 ms) : 0, 838655
BytebuddyAgent [candidate] (834.918 ms) : 0, 834918
GlobalTracer [baseline] (231.08 ms) : 0, 231080
GlobalTracer [candidate] (254.776 ms) : 0, 254776
AppSec [baseline] (57.568 ms) : 0, 57568
AppSec [candidate] (58.047 ms) : 0, 58047
IAST [baseline] (22.924 ms) : 0, 22924
IAST [candidate] (21.745 ms) : 0, 21745
Remote Config [baseline] (618.333 µs) : 0, 618
Remote Config [candidate] (668.601 µs) : 0, 669
Telemetry [baseline] (8.816 ms) : 0, 8816
Telemetry [candidate] (8.85 ms) : 0, 8850
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (837.429 ms) : 0, 837429
BytebuddyAgent [candidate] (832.669 ms) : 0, 832669
GlobalTracer [baseline] (230.604 ms) : 0, 230604
GlobalTracer [candidate] (252.848 ms) : 0, 252848
AppSec [baseline] (57.428 ms) : 0, 57428
AppSec [candidate] (57.922 ms) : 0, 57922
IAST [baseline] (22.995 ms) : 0, 22995
IAST [candidate] (21.584 ms) : 0, 21584
Remote Config [baseline] (607.792 µs) : 0, 608
Remote Config [candidate] (669.129 µs) : 0, 669
Telemetry [baseline] (8.694 ms) : 0, 8694
Telemetry [candidate] (8.798 ms) : 0, 8798
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (837.099 ms) : 0, 837099
BytebuddyAgent [candidate] (829.703 ms) : 0, 829703
GlobalTracer [baseline] (231.116 ms) : 0, 231116
GlobalTracer [candidate] (252.174 ms) : 0, 252174
AppSec [baseline] (52.529 ms) : 0, 52529
AppSec [candidate] (57.024 ms) : 0, 57024
IAST [baseline] (26.854 ms) : 0, 26854
IAST [candidate] (21.072 ms) : 0, 21072
Remote Config [baseline] (623.388 µs) : 0, 623
Remote Config [candidate] (662.282 µs) : 0, 662
Telemetry [baseline] (8.569 ms) : 0, 8569
Telemetry [candidate] (8.626 ms) : 0, 8626
Loading
Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1047175
Total [baseline] (10.391 s) : 0, 10390820
Agent [candidate] (1.062 s) : 0, 1061779
Total [candidate] (10.471 s) : 0, 10470755
section appsec
Agent [baseline] (1.189 s) : 0, 1188542
Total [baseline] (10.766 s) : 0, 10766367
Agent [candidate] (1.195 s) : 0, 1194880
Total [candidate] (10.776 s) : 0, 10775579
section iast
Agent [baseline] (1.187 s) : 0, 1186671
Total [baseline] (10.977 s) : 0, 10976879
Agent [candidate] (1.19 s) : 0, 1189794
Total [candidate] (10.969 s) : 0, 10969023
section profiling
Agent [baseline] (1.262 s) : 0, 1261601
Total [baseline] (10.934 s) : 0, 10933641
Agent [candidate] (1.279 s) : 0, 1278689
Total [candidate] (10.825 s) : 0, 10824805
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.047 s -
Agent appsec 1.189 s 141.367 ms (13.5%)
Agent iast 1.187 s 139.496 ms (13.3%)
Agent profiling 1.262 s 214.426 ms (20.5%)
Total tracing 10.391 s -
Total appsec 10.766 s 375.546 ms (3.6%)
Total iast 10.977 s 586.059 ms (5.6%)
Total profiling 10.934 s 542.82 ms (5.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.195 s 133.1 ms (12.5%)
Agent iast 1.19 s 128.014 ms (12.1%)
Agent profiling 1.279 s 216.91 ms (20.4%)
Total tracing 10.471 s -
Total appsec 10.776 s 304.823 ms (2.9%)
Total iast 10.969 s 498.268 ms (4.8%)
Total profiling 10.825 s 354.05 ms (3.4%)
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (722.272 ms) : 0, 722272
BytebuddyAgent [candidate] (714.766 ms) : 0, 714766
GlobalTracer [baseline] (240.913 ms) : 0, 240913
GlobalTracer [candidate] (261.536 ms) : 0, 261536
AppSec [baseline] (55.759 ms) : 0, 55759
AppSec [candidate] (58.392 ms) : 0, 58392
Remote Config [baseline] (697.006 µs) : 0, 697
Remote Config [candidate] (716.104 µs) : 0, 716
Telemetry [baseline] (12.255 ms) : 0, 12255
Telemetry [candidate] (11.305 ms) : 0, 11305
section appsec
BytebuddyAgent [baseline] (738.717 ms) : 0, 738717
BytebuddyAgent [candidate] (734.471 ms) : 0, 734471
GlobalTracer [baseline] (237.255 ms) : 0, 237255
GlobalTracer [candidate] (259.322 ms) : 0, 259322
AppSec [baseline] (177.495 ms) : 0, 177495
AppSec [candidate] (166.783 ms) : 0, 166783
IAST [baseline] (21.662 ms) : 0, 21662
IAST [candidate] (20.454 ms) : 0, 20454
Remote Config [baseline] (661.122 µs) : 0, 661
Remote Config [candidate] (665.765 µs) : 0, 666
Telemetry [baseline] (8.304 ms) : 0, 8304
Telemetry [candidate] (8.311 ms) : 0, 8311
section iast
BytebuddyAgent [baseline] (847.799 ms) : 0, 847799
BytebuddyAgent [candidate] (833.087 ms) : 0, 833087
GlobalTracer [baseline] (232.723 ms) : 0, 232723
GlobalTracer [candidate] (252.714 ms) : 0, 252714
AppSec [baseline] (57.926 ms) : 0, 57926
AppSec [candidate] (57.802 ms) : 0, 57802
IAST [baseline] (23.183 ms) : 0, 23183
IAST [candidate] (21.658 ms) : 0, 21658
Remote Config [baseline] (627.09 µs) : 0, 627
Remote Config [candidate] (666.147 µs) : 0, 666
Telemetry [baseline] (8.933 ms) : 0, 8933
Telemetry [candidate] (8.744 ms) : 0, 8744
section profiling
BytebuddyAgent [baseline] (708.568 ms) : 0, 708568
BytebuddyAgent [candidate] (705.662 ms) : 0, 705662
GlobalTracer [baseline] (349.755 ms) : 0, 349755
GlobalTracer [candidate] (374.284 ms) : 0, 374284
AppSec [baseline] (55.514 ms) : 0, 55514
AppSec [candidate] (53.809 ms) : 0, 53809
Remote Config [baseline] (665.79 µs) : 0, 666
Remote Config [candidate] (694.428 µs) : 0, 694
Telemetry [baseline] (8.864 ms) : 0, 8864
Telemetry [candidate] (8.769 ms) : 0, 8769
ProfilingAgent [baseline] (96.009 ms) : 0, 96009
ProfilingAgent [candidate] (93.514 ms) : 0, 93514
Profiling [baseline] (96.034 ms) : 0, 96034
Profiling [candidate] (93.538 ms) : 0, 93538
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-02-20T16:55:22 2025-02-20T17:02:25
git_branch master gary/add-llm-obs-writer
git_commit_date 1740058310 1740066638
git_commit_sha 10dc2ef 9ee77d9
release_version 1.47.0-SNAPSHOT~10dc2ef4fb 1.46.0-SNAPSHOT~9ee77d9810
start_time 2025-02-20T16:55:08 2025-02-20T17:02:11
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1740071299 1740071299
ci_job_id 816161626 816161626
ci_pipeline_id 56442606 56442606
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-ljefsgmc-project-304-concurrent-1-9cumdtra 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-ljefsgmc-project-304-concurrent-1-9cumdtra 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:iast_FULL better
[-89.230µs; -42.334µs] or [-12.312%; -5.841%]
unstable
[-665.894op/s; +1999.227op/s] or [-11.098%; +33.320%]
658.955µs 6666.667op/s 724.737µs 6000.000op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.372 ms) : 1352, 1392
.   : milestone, 1372,
appsec (1.76 ms) : 1737, 1783
.   : milestone, 1760,
appsec_no_iast (1.753 ms) : 1728, 1777
.   : milestone, 1753,
iast (1.519 ms) : 1496, 1543
.   : milestone, 1519,
profiling (1.511 ms) : 1487, 1535
.   : milestone, 1511,
tracing (1.485 ms) : 1459, 1511
.   : milestone, 1485,
section candidate
no_agent (1.353 ms) : 1333, 1372
.   : milestone, 1353,
appsec (1.734 ms) : 1710, 1758
.   : milestone, 1734,
appsec_no_iast (1.77 ms) : 1747, 1793
.   : milestone, 1770,
iast (1.507 ms) : 1484, 1529
.   : milestone, 1507,
profiling (1.506 ms) : 1483, 1529
.   : milestone, 1506,
tracing (1.495 ms) : 1471, 1519
.   : milestone, 1495,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.372 ms [1.352 ms, 1.392 ms] -
appsec 1.76 ms [1.737 ms, 1.783 ms] 387.974 µs (28.3%)
appsec_no_iast 1.753 ms [1.728 ms, 1.777 ms] 380.773 µs (27.8%)
iast 1.519 ms [1.496 ms, 1.543 ms] 147.245 µs (10.7%)
profiling 1.511 ms [1.487 ms, 1.535 ms] 139.083 µs (10.1%)
tracing 1.485 ms [1.459 ms, 1.511 ms] 112.856 µs (8.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.333 ms, 1.372 ms] -
appsec 1.734 ms [1.71 ms, 1.758 ms] 381.173 µs (28.2%)
appsec_no_iast 1.77 ms [1.747 ms, 1.793 ms] 417.267 µs (30.8%)
iast 1.507 ms [1.484 ms, 1.529 ms] 153.662 µs (11.4%)
profiling 1.506 ms [1.483 ms, 1.529 ms] 153.477 µs (11.3%)
tracing 1.495 ms [1.471 ms, 1.519 ms] 141.878 µs (10.5%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (379.494 µs) : 359, 400
.   : milestone, 379,
iast (510.35 µs) : 488, 532
.   : milestone, 510,
iast_FULL (724.737 µs) : 703, 747
.   : milestone, 725,
iast_GLOBAL (549.813 µs) : 528, 571
.   : milestone, 550,
iast_HARDCODED_SECRET_DISABLED (513.847 µs) : 491, 536
.   : milestone, 514,
iast_INACTIVE (456.09 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (498.331 µs) : 475, 522
.   : milestone, 498,
tracing (453.64 µs) : 432, 475
.   : milestone, 454,
section candidate
no_agent (385.176 µs) : 364, 406
.   : milestone, 385,
iast (498.494 µs) : 477, 520
.   : milestone, 498,
iast_FULL (658.955 µs) : 638, 680
.   : milestone, 659,
iast_GLOBAL (533.216 µs) : 511, 556
.   : milestone, 533,
iast_HARDCODED_SECRET_DISABLED (500.657 µs) : 479, 522
.   : milestone, 501,
iast_INACTIVE (465.978 µs) : 444, 487
.   : milestone, 466,
iast_TELEMETRY_OFF (484.914 µs) : 464, 506
.   : milestone, 485,
tracing (457.834 µs) : 437, 479
.   : milestone, 458,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 379.494 µs [359.329 µs, 399.66 µs] -
iast 510.35 µs [488.449 µs, 532.251 µs] 130.856 µs (34.5%)
iast_FULL 724.737 µs [702.599 µs, 746.875 µs] 345.243 µs (91.0%)
iast_GLOBAL 549.813 µs [528.188 µs, 571.439 µs] 170.319 µs (44.9%)
iast_HARDCODED_SECRET_DISABLED 513.847 µs [491.39 µs, 536.303 µs] 134.352 µs (35.4%)
iast_INACTIVE 456.09 µs [434.88 µs, 477.299 µs] 76.595 µs (20.2%)
iast_TELEMETRY_OFF 498.331 µs [474.812 µs, 521.85 µs] 118.836 µs (31.3%)
tracing 453.64 µs [432.33 µs, 474.95 µs] 74.146 µs (19.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 385.176 µs [364.326 µs, 406.026 µs] -
iast 498.494 µs [477.093 µs, 519.895 µs] 113.318 µs (29.4%)
iast_FULL 658.955 µs [637.518 µs, 680.392 µs] 273.779 µs (71.1%)
iast_GLOBAL 533.216 µs [510.869 µs, 555.564 µs] 148.04 µs (38.4%)
iast_HARDCODED_SECRET_DISABLED 500.657 µs [478.993 µs, 522.322 µs] 115.481 µs (30.0%)
iast_INACTIVE 465.978 µs [444.466 µs, 487.49 µs] 80.802 µs (21.0%)
iast_TELEMETRY_OFF 484.914 µs [463.579 µs, 506.249 µs] 99.738 µs (25.9%)
tracing 457.834 µs [437.02 µs, 478.648 µs] 72.658 µs (18.9%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master gary/add-llm-obs-writer
git_commit_date 1740058310 1740066638
git_commit_sha 10dc2ef 9ee77d9
release_version 1.47.0-SNAPSHOT~10dc2ef4fb 1.46.0-SNAPSHOT~9ee77d9810
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1740072610 1740072610
ci_job_id 816161627 816161627
ci_pipeline_id 56442606 56442606
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-bienyutl-project-304-concurrent-2-ymi0hyc4 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-bienyutl-project-304-concurrent-2-ymi0hyc4 6.8.0-1021-aws #23~22.04.1-Ubuntu SMP Tue Dec 10 16:50:46 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.289 s) : 15289000, 15289000
.   : milestone, 15289000,
appsec (14.915 s) : 14915000, 14915000
.   : milestone, 14915000,
iast (19.103 s) : 19103000, 19103000
.   : milestone, 19103000,
iast_GLOBAL (17.764 s) : 17764000, 17764000
.   : milestone, 17764000,
profiling (15.438 s) : 15438000, 15438000
.   : milestone, 15438000,
tracing (14.841 s) : 14841000, 14841000
.   : milestone, 14841000,
section candidate
no_agent (15.232 s) : 15232000, 15232000
.   : milestone, 15232000,
appsec (14.966 s) : 14966000, 14966000
.   : milestone, 14966000,
iast (18.526 s) : 18526000, 18526000
.   : milestone, 18526000,
iast_GLOBAL (18.027 s) : 18027000, 18027000
.   : milestone, 18027000,
profiling (15.161 s) : 15161000, 15161000
.   : milestone, 15161000,
tracing (14.951 s) : 14951000, 14951000
.   : milestone, 14951000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.289 s [15.289 s, 15.289 s] -
appsec 14.915 s [14.915 s, 14.915 s] -374.0 ms (-2.4%)
iast 19.103 s [19.103 s, 19.103 s] 3.814 s (24.9%)
iast_GLOBAL 17.764 s [17.764 s, 17.764 s] 2.475 s (16.2%)
profiling 15.438 s [15.438 s, 15.438 s] 149.0 ms (1.0%)
tracing 14.841 s [14.841 s, 14.841 s] -448.0 ms (-2.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.232 s [15.232 s, 15.232 s] -
appsec 14.966 s [14.966 s, 14.966 s] -266.0 ms (-1.7%)
iast 18.526 s [18.526 s, 18.526 s] 3.294 s (21.6%)
iast_GLOBAL 18.027 s [18.027 s, 18.027 s] 2.795 s (18.3%)
profiling 15.161 s [15.161 s, 15.161 s] -71.0 ms (-0.5%)
tracing 14.951 s [14.951 s, 14.951 s] -281.0 ms (-1.8%)
Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~9ee77d9810, baseline=1.47.0-SNAPSHOT~10dc2ef4fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1458, 1481
.   : milestone, 1469,
appsec (2.351 ms) : 2307, 2395
.   : milestone, 2351,
iast (2.098 ms) : 2043, 2153
.   : milestone, 2098,
iast_GLOBAL (2.143 ms) : 2088, 2198
.   : milestone, 2143,
profiling (1.952 ms) : 1908, 1995
.   : milestone, 1952,
tracing (1.941 ms) : 1898, 1983
.   : milestone, 1941,
section candidate
no_agent (1.464 ms) : 1452, 1475
.   : milestone, 1464,
appsec (2.335 ms) : 2292, 2378
.   : milestone, 2335,
iast (2.097 ms) : 2042, 2151
.   : milestone, 2097,
iast_GLOBAL (2.14 ms) : 2085, 2195
.   : milestone, 2140,
profiling (1.973 ms) : 1929, 2016
.   : milestone, 1973,
tracing (1.939 ms) : 1896, 1981
.   : milestone, 1939,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.458 ms, 1.481 ms] -
appsec 2.351 ms [2.307 ms, 2.395 ms] 881.799 µs (60.0%)
iast 2.098 ms [2.043 ms, 2.153 ms] 628.935 µs (42.8%)
iast_GLOBAL 2.143 ms [2.088 ms, 2.198 ms] 673.982 µs (45.9%)
profiling 1.952 ms [1.908 ms, 1.995 ms] 482.461 µs (32.8%)
tracing 1.941 ms [1.898 ms, 1.983 ms] 471.377 µs (32.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.464 ms [1.452 ms, 1.475 ms] -
appsec 2.335 ms [2.292 ms, 2.378 ms] 871.286 µs (59.5%)
iast 2.097 ms [2.042 ms, 2.151 ms] 632.883 µs (43.2%)
iast_GLOBAL 2.14 ms [2.085 ms, 2.195 ms] 676.59 µs (46.2%)
profiling 1.973 ms [1.929 ms, 2.016 ms] 508.895 µs (34.8%)
tracing 1.939 ms [1.896 ms, 1.981 ms] 474.936 µs (32.4%)

@gary-huang gary-huang force-pushed the gary/impl-llm-obs-api branch from b1d7aeb to 3bcbe81 Compare February 14, 2025 05:35
@gary-huang gary-huang force-pushed the gary/add-llm-obs-writer branch from 9052dc4 to dd8c1ed Compare February 14, 2025 09:13
@gary-huang gary-huang force-pushed the gary/impl-llm-obs-api branch from 3bcbe81 to c437b9c Compare February 19, 2025 14:47
@gary-huang gary-huang force-pushed the gary/add-llm-obs-writer branch 2 times, most recently from 88cbf86 to ce8a6d3 Compare February 19, 2025 17:32
@gary-huang gary-huang force-pushed the gary/impl-llm-obs-api branch from c437b9c to 8247b74 Compare February 19, 2025 17:50
@gary-huang gary-huang force-pushed the gary/add-llm-obs-writer branch 2 times, most recently from fc30660 to 061b18d Compare February 19, 2025 18:18
@gary-huang gary-huang force-pushed the gary/impl-llm-obs-api branch from 8247b74 to 871d358 Compare February 20, 2025 06:56
@gary-huang gary-huang force-pushed the gary/add-llm-obs-writer branch from 061b18d to 5c305b9 Compare February 20, 2025 06:57
@gary-huang gary-huang force-pushed the gary/add-llm-obs-writer branch from 5c305b9 to 9ee77d9 Compare February 20, 2025 15:50
private static class PayloadV2 extends Payload {
private static int size = 0;

private PayloadV2() {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Quality Violation

Consider adding super() or this() to your constructor (...read more)

In Java, it is suggested to call super() in an extended class. This rule will report a violation if both a call to super() and an overloaded constructor are absent.

View in Datadog  Leave us feedback  Documentation

@gary-huang gary-huang force-pushed the gary/impl-llm-obs-api branch 6 times, most recently from 94136c5 to 7064f57 Compare February 20, 2025 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant