Skip to content

Cache span.kind as byte ordinal for fast isOutbound()#11116

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 13 commits into
masterfrom
dougqh/cache-span-kind-ordinal
Apr 16, 2026
Merged

Cache span.kind as byte ordinal for fast isOutbound()#11116
gh-worker-dd-mergequeue-cf854d[bot] merged 13 commits into
masterfrom
dougqh/cache-span-kind-ordinal

Conversation

@dougqh
Copy link
Copy Markdown
Contributor

@dougqh dougqh commented Apr 15, 2026

What Does This Do

  • Caches span.kind as a volatile byte ordinal on DDSpanContext, following the existing httpStatusCode dual-store pattern
  • Rewrites DDSpan.isOutbound() from getTag() + String.equals() to a constant-time byte comparison (~2.8ns vs ~10ns)
  • Adds SPAN_KIND interception in TagInterceptor to populate the cache on setTag()
  • Includes IsOutboundBenchmark JMH benchmark

Motivation

DDSpanContext.getTag("span.kind") consumed ~14% of foreground CPU in a 16-thread span creation stress test. It was called from CoreTracer.onRootSpanStarted and onRootSpanFinished on every root span, falling through the getTag() switch to a full TagMap hash-table lookup with potential synchronization.

In spring petclinic throughput test, this result in 10-20% reduction in our throughput overhead with ample heap. Reducing throughput impact from adding tracer from -10% to -8%.

Additional Notes

Benchmark Master (before) PR #11116 (after) Speedup
isOutbound_client 8.929 ± 0.295 ns 2.857 ± 0.243 ns 3.1x
isOutbound_server 8.947 ± 0.116 ns 2.884 ± 0.155 ns 3.1x
isOutbound_unset 9.074 ± 0.055 ns 2.851 ± 0.160 ns 3.2x
getTag_spanKind_client 9.049 ± 0.972 ns 3.974 ± 0.070 ns 2.3x
getTag_spanKind_unset 8.859 ± 0.135 ns 9.753 ± 0.104 ns ~same (fallthrough)

Contributor Checklist

🤖 Generated with Claude Code

DDSpanContext.getTag("span.kind") was consuming ~14% of foreground CPU
in span creation stress tests. It was called from DDSpan.isOutbound()
on every root span start and finish, falling through the getTag() switch
to a full TagMap hash-table lookup with potential synchronization.

This change caches span.kind as a volatile byte ordinal on DDSpanContext
(same dual-store pattern as httpStatusCode). The TagInterceptor now
intercepts SPAN_KIND to set the ordinal, and isOutbound() does a simple
byte comparison instead of getTag() + String.equals(). Benchmark shows
isOutbound() at constant ~2.8ns regardless of span kind.

tag: no release note
tag: ai generated

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
- Reduce visibility of SPAN_KIND_UNSET and SPAN_KIND_CUSTOM to
  package-private (they don't correspond to actual span.kind values)
- Restructure setSpanKind: identity checks first as a fast path,
  then string switch fallback for non-interned strings. Extracted
  into static spanKindToOrdinal() helper.
- Use identity-first comparison in removeTag() for SPAN_KIND check
- Use ordinal > SPAN_KIND_UNSET && ordinal < SPAN_KIND_CUSTOM in
  getTag() to help JIT with bounds check elimination
- Use Tags.SPAN_KIND constant instead of key in fallthrough unsafeGetTag

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Apr 15, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dougqh/cache-span-kind-ordinal
git_commit_date 1776364287 1776370079
git_commit_sha 3fb3733 b957f87
release_version 1.62.0-SNAPSHOT~3fb37337ab 1.62.0-SNAPSHOT~b957f8718c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776371800 1776371800
ci_job_id 1603478150 1603478150
ci_pipeline_id 108116838 108116838
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-2fcwu9mg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-2fcwu9mg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for petclinic
gantt
    title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059403
Total [baseline] (11.102 s) : 0, 11101790
Agent [candidate] (1.066 s) : 0, 1065872
Total [candidate] (11.249 s) : 0, 11248943
section appsec
Agent [baseline] (1.258 s) : 0, 1257946
Total [baseline] (11.219 s) : 0, 11218560
Agent [candidate] (1.246 s) : 0, 1245669
Total [candidate] (11.093 s) : 0, 11092808
section iast
Agent [baseline] (1.233 s) : 0, 1233347
Total [baseline] (11.327 s) : 0, 11326947
Agent [candidate] (1.227 s) : 0, 1226949
Total [candidate] (11.379 s) : 0, 11379317
section profiling
Agent [baseline] (1.202 s) : 0, 1201927
Total [baseline] (11.038 s) : 0, 11037736
Agent [candidate] (1.186 s) : 0, 1186292
Total [candidate] (11.089 s) : 0, 11088747
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.258 s 198.543 ms (18.7%)
Agent iast 1.233 s 173.944 ms (16.4%)
Agent profiling 1.202 s 142.523 ms (13.5%)
Total tracing 11.102 s -
Total appsec 11.219 s 116.77 ms (1.1%)
Total iast 11.327 s 225.157 ms (2.0%)
Total profiling 11.038 s -64.054 ms (-0.6%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent appsec 1.246 s 179.797 ms (16.9%)
Agent iast 1.227 s 161.077 ms (15.1%)
Agent profiling 1.186 s 120.42 ms (11.3%)
Total tracing 11.249 s -
Total appsec 11.093 s -156.135 ms (-1.4%)
Total iast 11.379 s 130.374 ms (1.2%)
Total profiling 11.089 s -160.195 ms (-1.4%)
gantt
    title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.227 ms) : 0, 1227
crashtracking [candidate] (1.233 ms) : 0, 1233
BytebuddyAgent [baseline] (633.111 ms) : 0, 633111
BytebuddyAgent [candidate] (637.772 ms) : 0, 637772
AgentMeter [baseline] (29.506 ms) : 0, 29506
AgentMeter [candidate] (29.651 ms) : 0, 29651
GlobalTracer [baseline] (249.16 ms) : 0, 249160
GlobalTracer [candidate] (250.446 ms) : 0, 250446
AppSec [baseline] (32.373 ms) : 0, 32373
AppSec [candidate] (32.486 ms) : 0, 32486
Debugger [baseline] (59.991 ms) : 0, 59991
Debugger [candidate] (60.349 ms) : 0, 60349
Remote Config [baseline] (598.77 µs) : 0, 599
Remote Config [candidate] (595.081 µs) : 0, 595
Telemetry [baseline] (8.106 ms) : 0, 8106
Telemetry [candidate] (8.134 ms) : 0, 8134
Flare Poller [baseline] (9.111 ms) : 0, 9111
Flare Poller [candidate] (8.879 ms) : 0, 8879
section appsec
crashtracking [baseline] (1.236 ms) : 0, 1236
crashtracking [candidate] (1.216 ms) : 0, 1216
BytebuddyAgent [baseline] (667.289 ms) : 0, 667289
BytebuddyAgent [candidate] (659.877 ms) : 0, 659877
AgentMeter [baseline] (12.162 ms) : 0, 12162
AgentMeter [candidate] (12.089 ms) : 0, 12089
GlobalTracer [baseline] (250.754 ms) : 0, 250754
GlobalTracer [candidate] (248.754 ms) : 0, 248754
IAST [baseline] (24.634 ms) : 0, 24634
IAST [candidate] (24.527 ms) : 0, 24527
AppSec [baseline] (186.153 ms) : 0, 186153
AppSec [candidate] (185.014 ms) : 0, 185014
Debugger [baseline] (66.362 ms) : 0, 66362
Debugger [candidate] (65.405 ms) : 0, 65405
Remote Config [baseline] (618.465 µs) : 0, 618
Remote Config [candidate] (604.228 µs) : 0, 604
Telemetry [baseline] (8.487 ms) : 0, 8487
Telemetry [candidate] (8.326 ms) : 0, 8326
Flare Poller [baseline] (3.565 ms) : 0, 3565
Flare Poller [candidate] (3.483 ms) : 0, 3483
section iast
crashtracking [baseline] (1.27 ms) : 0, 1270
crashtracking [candidate] (1.238 ms) : 0, 1238
BytebuddyAgent [baseline] (808.818 ms) : 0, 808818
BytebuddyAgent [candidate] (803.135 ms) : 0, 803135
AgentMeter [baseline] (11.599 ms) : 0, 11599
AgentMeter [candidate] (11.428 ms) : 0, 11428
GlobalTracer [baseline] (239.531 ms) : 0, 239531
GlobalTracer [candidate] (239.891 ms) : 0, 239891
IAST [baseline] (25.768 ms) : 0, 25768
IAST [candidate] (26.496 ms) : 0, 26496
AppSec [baseline] (31.335 ms) : 0, 31335
AppSec [candidate] (31.936 ms) : 0, 31936
Debugger [baseline] (65.265 ms) : 0, 65265
Debugger [candidate] (63.278 ms) : 0, 63278
Remote Config [baseline] (541.514 µs) : 0, 542
Remote Config [candidate] (537.877 µs) : 0, 538
Telemetry [baseline] (9.399 ms) : 0, 9399
Telemetry [candidate] (9.276 ms) : 0, 9276
Flare Poller [baseline] (3.545 ms) : 0, 3545
Flare Poller [candidate] (3.535 ms) : 0, 3535
section profiling
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.18 ms) : 0, 1180
BytebuddyAgent [baseline] (703.394 ms) : 0, 703394
BytebuddyAgent [candidate] (692.545 ms) : 0, 692545
AgentMeter [baseline] (9.203 ms) : 0, 9203
AgentMeter [candidate] (9.106 ms) : 0, 9106
GlobalTracer [baseline] (209.687 ms) : 0, 209687
GlobalTracer [candidate] (207.283 ms) : 0, 207283
AppSec [baseline] (33.207 ms) : 0, 33207
AppSec [candidate] (32.763 ms) : 0, 32763
Debugger [baseline] (66.387 ms) : 0, 66387
Debugger [candidate] (65.677 ms) : 0, 65677
Remote Config [baseline] (581.231 µs) : 0, 581
Remote Config [candidate] (578.127 µs) : 0, 578
Telemetry [baseline] (7.85 ms) : 0, 7850
Telemetry [candidate] (7.855 ms) : 0, 7855
Flare Poller [baseline] (3.572 ms) : 0, 3572
Flare Poller [candidate] (3.59 ms) : 0, 3590
ProfilingAgent [baseline] (94.795 ms) : 0, 94795
ProfilingAgent [candidate] (94.394 ms) : 0, 94394
Profiling [baseline] (95.374 ms) : 0, 95374
Profiling [candidate] (94.955 ms) : 0, 94955
Loading
Startup time reports for insecure-bank
gantt
    title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064010
Total [baseline] (8.894 s) : 0, 8893631
Agent [candidate] (1.057 s) : 0, 1057368
Total [candidate] (8.858 s) : 0, 8858345
section iast
Agent [baseline] (1.224 s) : 0, 1223518
Total [baseline] (9.549 s) : 0, 9549271
Agent [candidate] (1.224 s) : 0, 1224123
Total [candidate] (9.551 s) : 0, 9551067
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.224 s 159.508 ms (15.0%)
Total tracing 8.894 s -
Total iast 9.549 s 655.64 ms (7.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.057 s -
Agent iast 1.224 s 166.755 ms (15.8%)
Total tracing 8.858 s -
Total iast 9.551 s 692.723 ms (7.8%)
gantt
    title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.252 ms) : 0, 1252
crashtracking [candidate] (1.232 ms) : 0, 1232
BytebuddyAgent [baseline] (635.636 ms) : 0, 635636
BytebuddyAgent [candidate] (633.046 ms) : 0, 633046
AgentMeter [baseline] (29.789 ms) : 0, 29789
AgentMeter [candidate] (29.306 ms) : 0, 29306
GlobalTracer [baseline] (250.863 ms) : 0, 250863
GlobalTracer [candidate] (249.14 ms) : 0, 249140
AppSec [baseline] (32.646 ms) : 0, 32646
AppSec [candidate] (32.333 ms) : 0, 32333
Debugger [baseline] (59.615 ms) : 0, 59615
Debugger [candidate] (59.161 ms) : 0, 59161
Remote Config [baseline] (602.012 µs) : 0, 602
Remote Config [candidate] (600.092 µs) : 0, 600
Telemetry [baseline] (8.13 ms) : 0, 8130
Telemetry [candidate] (8.1 ms) : 0, 8100
Flare Poller [baseline] (9.079 ms) : 0, 9079
Flare Poller [candidate] (8.313 ms) : 0, 8313
section iast
crashtracking [baseline] (1.242 ms) : 0, 1242
crashtracking [candidate] (1.226 ms) : 0, 1226
BytebuddyAgent [baseline] (800.541 ms) : 0, 800541
BytebuddyAgent [candidate] (801.146 ms) : 0, 801146
AgentMeter [baseline] (11.424 ms) : 0, 11424
AgentMeter [candidate] (11.456 ms) : 0, 11456
GlobalTracer [baseline] (238.623 ms) : 0, 238623
GlobalTracer [candidate] (239.249 ms) : 0, 239249
AppSec [baseline] (33.763 ms) : 0, 33763
AppSec [candidate] (33.83 ms) : 0, 33830
Debugger [baseline] (62.115 ms) : 0, 62115
Debugger [candidate] (61.921 ms) : 0, 61921
Remote Config [baseline] (556.725 µs) : 0, 557
Remote Config [candidate] (546.748 µs) : 0, 547
Telemetry [baseline] (9.508 ms) : 0, 9508
Telemetry [candidate] (9.289 ms) : 0, 9289
Flare Poller [baseline] (3.699 ms) : 0, 3699
Flare Poller [candidate] (3.556 ms) : 0, 3556
IAST [baseline] (25.831 ms) : 0, 25831
IAST [candidate] (25.772 ms) : 0, 25772
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dougqh/cache-span-kind-ordinal
git_commit_date 1776364287 1776370079
git_commit_sha 3fb3733 b957f87
release_version 1.62.0-SNAPSHOT~3fb37337ab 1.62.0-SNAPSHOT~b957f8718c
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1776372367 1776372367
ci_job_id 1603478152 1603478152
ci_pipeline_id 108116838 108116838
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-d22h9is5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-d22h9is5 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 3 performance regressions! Performance is the same for 18 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:petclinic:appsec:high_load worse
[+0.608ms; +1.937ms] or [+3.295%; +10.503%]
unsure
[+0.408ms; +1.987ms] or [+1.359%; +6.618%]
unstable
[-38.104op/s; +12.167op/s] or [-15.295%; +4.884%]
19.710ms 31.225ms 236.156op/s 18.438ms 30.028ms 249.125op/s
scenario:load:petclinic:no_agent:high_load worse
[+1.532ms; +2.615ms] or [+9.233%; +15.757%]
worse
[+1.139ms; +3.829ms] or [+4.008%; +13.477%]
unstable
[-53.095op/s; +0.783op/s] or [-19.604%; +0.289%]
18.670ms 30.894ms 244.688op/s 16.596ms 28.410ms 270.844op/s
Request duration reports for petclinic
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (17.222 ms) : 17051, 17394
.   : milestone, 17222,
appsec (18.733 ms) : 18544, 18923
.   : milestone, 18733,
code_origins (17.932 ms) : 17755, 18110
.   : milestone, 17932,
iast (17.882 ms) : 17707, 18058
.   : milestone, 17882,
profiling (18.118 ms) : 17940, 18296
.   : milestone, 18118,
tracing (17.779 ms) : 17605, 17953
.   : milestone, 17779,
section candidate
no_agent (19.075 ms) : 18881, 19269
.   : milestone, 19075,
appsec (19.764 ms) : 19562, 19966
.   : milestone, 19764,
code_origins (17.906 ms) : 17728, 18085
.   : milestone, 17906,
iast (17.728 ms) : 17554, 17901
.   : milestone, 17728,
profiling (18.238 ms) : 18059, 18418
.   : milestone, 18238,
tracing (17.835 ms) : 17659, 18010
.   : milestone, 17835,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 17.222 ms [17.051 ms, 17.394 ms] -
appsec 18.733 ms [18.544 ms, 18.923 ms] 1.511 ms (8.8%)
code_origins 17.932 ms [17.755 ms, 18.11 ms] 710.147 µs (4.1%)
iast 17.882 ms [17.707 ms, 18.058 ms] 660.0 µs (3.8%)
profiling 18.118 ms [17.94 ms, 18.296 ms] 895.811 µs (5.2%)
tracing 17.779 ms [17.605 ms, 17.953 ms] 557.06 µs (3.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 19.075 ms [18.881 ms, 19.269 ms] -
appsec 19.764 ms [19.562 ms, 19.966 ms] 688.997 µs (3.6%)
code_origins 17.906 ms [17.728 ms, 18.085 ms] -1.169 ms (-6.1%)
iast 17.728 ms [17.554 ms, 17.901 ms] -1.347 ms (-7.1%)
profiling 18.238 ms [18.059 ms, 18.418 ms] -836.557 µs (-4.4%)
tracing 17.835 ms [17.659 ms, 18.01 ms] -1.24 ms (-6.5%)
Request duration reports for insecure-bank
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.247 ms) : 1235, 1260
.   : milestone, 1247,
iast (3.288 ms) : 3246, 3330
.   : milestone, 3288,
iast_FULL (6.309 ms) : 6244, 6374
.   : milestone, 6309,
iast_GLOBAL (3.673 ms) : 3610, 3736
.   : milestone, 3673,
profiling (2.241 ms) : 2220, 2261
.   : milestone, 2241,
tracing (1.928 ms) : 1911, 1945
.   : milestone, 1928,
section candidate
no_agent (1.254 ms) : 1242, 1265
.   : milestone, 1254,
iast (3.236 ms) : 3194, 3279
.   : milestone, 3236,
iast_FULL (6.436 ms) : 6367, 6505
.   : milestone, 6436,
iast_GLOBAL (3.604 ms) : 3544, 3664
.   : milestone, 3604,
profiling (2.366 ms) : 2342, 2390
.   : milestone, 2366,
tracing (1.84 ms) : 1825, 1854
.   : milestone, 1840,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.247 ms [1.235 ms, 1.26 ms] -
iast 3.288 ms [3.246 ms, 3.33 ms] 2.041 ms (163.6%)
iast_FULL 6.309 ms [6.244 ms, 6.374 ms] 5.062 ms (405.9%)
iast_GLOBAL 3.673 ms [3.61 ms, 3.736 ms] 2.426 ms (194.5%)
profiling 2.241 ms [2.22 ms, 2.261 ms] 993.545 µs (79.7%)
tracing 1.928 ms [1.911 ms, 1.945 ms] 681.193 µs (54.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.254 ms [1.242 ms, 1.265 ms] -
iast 3.236 ms [3.194 ms, 3.279 ms] 1.983 ms (158.2%)
iast_FULL 6.436 ms [6.367 ms, 6.505 ms] 5.182 ms (413.4%)
iast_GLOBAL 3.604 ms [3.544 ms, 3.664 ms] 2.351 ms (187.5%)
profiling 2.366 ms [2.342 ms, 2.39 ms] 1.113 ms (88.8%)
tracing 1.84 ms [1.825 ms, 1.854 ms] 586.083 µs (46.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dougqh/cache-span-kind-ordinal
git_commit_date 1776364287 1776370079
git_commit_sha 3fb3733 b957f87
release_version 1.62.0-SNAPSHOT~3fb37337ab 1.62.0-SNAPSHOT~b957f8718c
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1776372087 1776372087
ci_job_id 1603478153 1603478153
ci_pipeline_id 108116838 108116838
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-4-ifuls8xg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-4-ifuls8xg 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.488 ms) : 1477, 1500
.   : milestone, 1488,
appsec (2.547 ms) : 2492, 2602
.   : milestone, 2547,
iast (2.265 ms) : 2196, 2333
.   : milestone, 2265,
iast_GLOBAL (2.311 ms) : 2241, 2380
.   : milestone, 2311,
profiling (2.097 ms) : 2042, 2151
.   : milestone, 2097,
tracing (2.077 ms) : 2023, 2130
.   : milestone, 2077,
section candidate
no_agent (1.488 ms) : 1476, 1499
.   : milestone, 1488,
appsec (3.844 ms) : 3620, 4067
.   : milestone, 3844,
iast (2.27 ms) : 2200, 2339
.   : milestone, 2270,
iast_GLOBAL (2.314 ms) : 2245, 2384
.   : milestone, 2314,
profiling (2.101 ms) : 2046, 2156
.   : milestone, 2101,
tracing (2.09 ms) : 2036, 2143
.   : milestone, 2090,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.488 ms [1.477 ms, 1.5 ms] -
appsec 2.547 ms [2.492 ms, 2.602 ms] 1.058 ms (71.1%)
iast 2.265 ms [2.196 ms, 2.333 ms] 776.306 µs (52.2%)
iast_GLOBAL 2.311 ms [2.241 ms, 2.38 ms] 822.169 µs (55.2%)
profiling 2.097 ms [2.042 ms, 2.151 ms] 608.5 µs (40.9%)
tracing 2.077 ms [2.023 ms, 2.13 ms] 588.238 µs (39.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.488 ms [1.476 ms, 1.499 ms] -
appsec 3.844 ms [3.62 ms, 4.067 ms] 2.356 ms (158.4%)
iast 2.27 ms [2.2 ms, 2.339 ms] 781.84 µs (52.6%)
iast_GLOBAL 2.314 ms [2.245 ms, 2.384 ms] 826.499 µs (55.6%)
profiling 2.101 ms [2.046 ms, 2.156 ms] 613.523 µs (41.2%)
tracing 2.09 ms [2.036 ms, 2.143 ms] 602.182 µs (40.5%)
Execution time for biojava
gantt
    title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~b957f8718c, baseline=1.62.0-SNAPSHOT~3fb37337ab
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.48 s) : 15480000, 15480000
.   : milestone, 15480000,
appsec (14.68 s) : 14680000, 14680000
.   : milestone, 14680000,
iast (18.243 s) : 18243000, 18243000
.   : milestone, 18243000,
iast_GLOBAL (18.098 s) : 18098000, 18098000
.   : milestone, 18098000,
profiling (14.773 s) : 14773000, 14773000
.   : milestone, 14773000,
tracing (15.237 s) : 15237000, 15237000
.   : milestone, 15237000,
section candidate
no_agent (14.912 s) : 14912000, 14912000
.   : milestone, 14912000,
appsec (14.722 s) : 14722000, 14722000
.   : milestone, 14722000,
iast (18.693 s) : 18693000, 18693000
.   : milestone, 18693000,
iast_GLOBAL (18.079 s) : 18079000, 18079000
.   : milestone, 18079000,
profiling (14.904 s) : 14904000, 14904000
.   : milestone, 14904000,
tracing (15.022 s) : 15022000, 15022000
.   : milestone, 15022000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.48 s [15.48 s, 15.48 s] -
appsec 14.68 s [14.68 s, 14.68 s] -800.0 ms (-5.2%)
iast 18.243 s [18.243 s, 18.243 s] 2.763 s (17.8%)
iast_GLOBAL 18.098 s [18.098 s, 18.098 s] 2.618 s (16.9%)
profiling 14.773 s [14.773 s, 14.773 s] -707.0 ms (-4.6%)
tracing 15.237 s [15.237 s, 15.237 s] -243.0 ms (-1.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.912 s [14.912 s, 14.912 s] -
appsec 14.722 s [14.722 s, 14.722 s] -190.0 ms (-1.3%)
iast 18.693 s [18.693 s, 18.693 s] 3.781 s (25.4%)
iast_GLOBAL 18.079 s [18.079 s, 18.079 s] 3.167 s (21.2%)
profiling 14.904 s [14.904 s, 14.904 s] -8.0 ms (-0.1%)
tracing 15.022 s [15.022 s, 15.022 s] 110.0 ms (0.7%)

Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
}

public void removeTag(String tag) {
if (tag == Tags.SPAN_KIND || Tags.SPAN_KIND.equals(tag)) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use equal helper function proposed in comment above

Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Comment thread dd-trace-core/src/jmh/java/datadog/trace/core/IsOutboundBenchmark.java Outdated
Comment thread dd-trace-core/src/jmh/java/datadog/trace/core/IsOutboundBenchmark.java Outdated
- Replace identity-first + string-switch with simpler if/else chain
  using a static tagEquals(tagValue, tagLiteral) helper that does
  identity check then falls back to equals
- Make all ordinal constants package-private
- Add isClientSpanKind() convenience method so TagInterceptor doesn't
  need to reference ordinal constants from a sub-package
- Use tagEquals helper in removeTag for consistency

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
- Extract SPAN_KIND getTag fallthrough into private getSpanKindTag()
  helper method for readability
- Simplify benchmark setup: cast to DDSpan immediately instead of
  using intermediate AgentSpan variables

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
…tring

- Remove superfluous isClientSpanKind() (only one call site, inline
  the check in TagInterceptor using string comparison instead)
- Reduce getSpanKindOrdinal() visibility to package-private
- Rename getSpanKindTag -> getSpanKindString, return String, use
  unsafeTags.getString for the fallthrough path

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
}
if (path != null) {
final boolean isClient = Tags.SPAN_KIND_CLIENT.equals(span.unsafeGetTag(Tags.SPAN_KIND));
final boolean isClient = Tags.SPAN_KIND_CLIENT.equals(span.getSpanKindString());
Copy link
Copy Markdown
Contributor Author

@dougqh dougqh Apr 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I decided to keep the "enum" package visible for now, we cannot do a faster ordinal comparison here. I think that's okay.

@dougqh dougqh added tag: ai generated Largely based on code generated by an AI or LLM type: enhancement Enhancements and improvements tag: performance Performance related changes comp: core Tracer core labels Apr 15, 2026
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java
@dougqh dougqh marked this pull request as ready for review April 15, 2026 16:12
@dougqh dougqh requested a review from a team as a code owner April 15, 2026 16:12
@dougqh dougqh requested a review from mcculls April 15, 2026 16:12

/** Identity-first string comparison: checks reference equality, then falls back to equals. */
static boolean tagEquals(String tagValue, String tagLiteral) {
return (tagValue == tagLiteral) || tagLiteral.equals(tagValue);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bit surprised this makes a difference, given String.equals has this == anObject as the first check :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the difference is usually pretty negligible especially when the the type is final and the call gets statically devirtualized.

I did see a small improvement from the same things for keys in TagMap, but don't recall how much.

public void setSpanKind(String kind) {
if (kind == null) {
spanKindOrdinal = SPAN_KIND_UNSET;
} else if (tagEquals(kind, Tags.SPAN_KIND_SERVER)) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does a string-switch perform compared to if...else ? I'd imagine the hashed jump would help here...

Copy link
Copy Markdown
Contributor Author

@dougqh dougqh Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I had Claude try that. It didn't make a much difference either way.
Admittedly, that was a fast path of referential checks followed by a slow path of string switch.

In past benchmarks, I've found that the fastest option is usually...
int hash = str.hashCode();
if ( hash == LITERAL_HASH && LITERAL.equals(str) ) {
...
}

That is close to what string switch does, but string switch is slower.
I also expect that the benefit of hashCode comparison can be situational.

For tags, we're often in the situation where the tag parameter is a constant, so the JIT can take advantage of inlining and constant propagation to optimize a lot. And I suspect that the hash comparison would likely get in the way of that.

But when the parameter isn't a constant, the hash comparison can clearly be beneficial.

Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java
Comment thread dd-trace-core/src/main/java/datadog/trace/core/DDSpanContext.java Outdated
Copy link
Copy Markdown
Contributor

@mcculls mcculls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some minor suggestions, nothing blocking

dougqh and others added 5 commits April 16, 2026 14:57
The method sets the cached ordinal, not the string form.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…stay in sync

Verifies that:
- SPAN_KIND_VALUES array covers all ordinals up to SPAN_KIND_CUSTOM
- Each known ordinal maps to the correct Tags.SPAN_KIND_* constant
- UNSET and CUSTOM map to null
- setSpanKindOrdinal round-trips correctly for all known kinds

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
Tests the public API path through TagInterceptor:
- setTag + getTag round-trip for all known span kinds
- getTag returns null when span.kind is not set
- removeTag clears span.kind
- custom (non-standard) span.kind falls back to tag map

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
@dougqh dougqh enabled auto-merge April 16, 2026 20:19
@dougqh dougqh added this pull request to the merge queue Apr 16, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Apr 16, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented Apr 16, 2026

View all feedbacks in Devflow UI.

2026-04-16 20:57:56 UTC ℹ️ Start processing command /merge


2026-04-16 20:58:01 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).


2026-04-16 22:04:17 UTC ℹ️ MergeQueue: This merge request was merged

@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 16, 2026
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 06b104b into master Apr 16, 2026
570 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the dougqh/cache-span-kind-ordinal branch April 16, 2026 22:04
@github-actions github-actions Bot added this to the 1.62.0 milestone Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: enhancement Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants