Skip to content

feat: emit TaskBlock events for JVM blocking states#665

Draft
kaahos wants to merge 14 commits into
mainfrom
paul.fournillon/taskblock-jvm-producers
Draft

feat: emit TaskBlock events for JVM blocking states#665
kaahos wants to merge 14 commits into
mainfrom
paul.fournillon/taskblock-jvm-producers

Conversation

@kaahos

@kaahos kaahos commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?:

Adds JVM TaskBlock producers for LockSupport.park and monitor contention/wait states, wiring JVM state transitions into the synchronous TaskBlock recording API.

Motivation:

Common JVM blocking operations can be represented as TaskBlock events instead of relying on wall-clock signals against blocked threads.

Additional Notes:

Virtual threads are not explicitly emitted by this PR. This change focuses on platform-thread JVM blocking states exposed through the existing VM entry paths.

How to test the change?:

Automated coverage includes native park-state tests and Java integration tests for park, monitor wait, monitor contention, JVMTI-backed variants, and combined wall-clock mitigation behavior.

  • ./.claude/commands/build-and-summarize testDebug

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a security review (run the dd:platform-security-review skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: [JIRA-XXXX]

@dd-octo-sts

dd-octo-sts Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #29839659489 | Commit: 6f42c70 | Duration: 17m 7s (longest job)

All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - - -
8-ibm - - -
8-j9 - -
8-librca - -
8-orcl - - -
11 - - -
11-j9 - -
11-librca - -
17 - -
17-graal - -
17-j9 - -
17-librca - -
21 - -
21-graal - -
21-librca - -
25 - -
25-graal - -
25-librca - -

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-07-21 14:56:17 UTC

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch 2 times, most recently from 1beeda2 to 38efbf8 Compare July 16, 2026 15:20
@dd-octo-sts

dd-octo-sts Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 38efbf8)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125051342 Commit: 38efbf8f7d94004f2ec282bc6c50148a99f67616

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +4.1% (2719→2830 ms)
  • 🔴 fj-kmeans (JDK 25): runtime +2.8% (2737→2815 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10365 ms (21 iters) ✅ 10195 ms (21 iters) ≈ -1.6% (±11.4%) — / —
akka-uct 25 ✅ 8859 ms (24 iters) ✅ 8925 ms (24 iters) ≈ +0.7% (±10.1%) — / —
finagle-chirper 21 ✅ 5945 ms (33 iters) ✅ 5953 ms (33 iters) ≈ +0.1% (±24.9%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5495 ms (36 iters) ✅ 5431 ms (36 iters) ≈ -1.2% (±24.2%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2719 ms (68 iters) ✅ 2830 ms (66 iters) 🔴 +4.1% — / —
fj-kmeans 25 ✅ 2737 ms (68 iters) ✅ 2815 ms (66 iters) 🔴 +2.8% — / —
future-genetic 21 ✅ 2114 ms (88 iters) ✅ 2140 ms (87 iters) ≈ +1.2% (±2.7%) — / —
future-genetic 25 ✅ 2080 ms (90 iters) ✅ 2126 ms (88 iters) ≈ +2.2% (±2.5%) — / —
naive-bayes 21 ✅ 1299 ms (131 iters) ✅ 1232 ms (139 iters) ≈ -5.2% (±31.9%) — / —
naive-bayes 25 ✅ 997 ms (172 iters) ✅ 1004 ms (170 iters) ≈ +0.7% (±31.8%) — / —
reactors 21 ✅ 16202 ms (15 iters) ✅ 15908 ms (15 iters) ≈ -1.8% (±7.6%) — / —
reactors 25 ✅ 18524 ms (15 iters) ✅ 18484 ms (15 iters) ≈ -0.2% (±3.9%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 2 / ✅ 2045 / 1958 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ ✅ / 3 2353 / 2059 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 6 / 2 8437 / 8382 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ ✅ / 2 8800 / 8439 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 2 / 2 1248 / 1279 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 4 / 1 1262 / 1271 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 2963 / 2980 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 3 2955 / 2936 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 3 / 7 3466 / 3539 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 5 / 1 3478 / 3531 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1700 / 1609 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1905 / 1873 ✅ / ✅ ✅ / ✅

@dd-octo-sts

dd-octo-sts Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Reliability & Chaos Results

All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/125035442

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from 38efbf8 to cdb8274 Compare July 16, 2026 22:13
@dd-octo-sts

dd-octo-sts Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit cdb8274)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125162043 Commit: cdb8274124fc1947560c5c91bbf213a248e6b608

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +5.1% (2683→2821 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
finagle-chirper 21 ✅ 5956 ms (33 iters) ✅ 6031 ms (33 iters) ≈ +1.3% (±24.8%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5457 ms (36 iters) ✅ 5444 ms (36 iters) ≈ -0.2% (±23.9%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2683 ms (70 iters) ✅ 2821 ms (66 iters) 🔴 +5.1% — / —
fj-kmeans 25 ✅ 2786 ms (67 iters) ✅ 2839 ms (66 iters) ≈ +1.9% (±2.7%) — / —
future-genetic 21 ✅ 2063 ms (90 iters) ✅ 2095 ms (89 iters) ≈ +1.6% (±2.6%) — / —
future-genetic 25 ✅ 2072 ms (90 iters) ✅ 2125 ms (87 iters) ≈ +2.6% (±2.6%) — / —
naive-bayes 21 ✅ 1257 ms (135 iters) ✅ 1250 ms (136 iters) ≈ -0.6% (±32.8%) — / —
reactors 21 ✅ 16137 ms (15 iters) ✅ 15974 ms (15 iters) ≈ -1% (±7.5%) — / —
reactors 25 ✅ 18395 ms (15 iters) ✅ 18505 ms (15 iters) ≈ +0.6% (±4.4%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 1 / 1 2260 / 2182 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 4 / 3 8741 / 8673 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8238 / 8373 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 4 / 2 1241 / 1256 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / ✅ 1287 / 1282 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 2 / 1 2971 / 2981 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 1 / 1 2968 / 2956 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 5 / 3 3468 / 3474 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 9 / 3 3464 / 3465 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 1 / 2 1531 / 1536 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 1857 / 1856 ✅ / ✅ ✅ / ✅

@dd-octo-sts

dd-octo-sts Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit e13ec27)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125211559 Commit: e13ec275311412f74c94ccb558f2c544418a0634

✅ Within expected boundaries

No significant runtime deltas (all within run-to-run noise) and no internal-counter outliers.

Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10495 ms (21 iters) ✅ 10316 ms (21 iters) ≈ -1.7% (±10.8%) — / —
akka-uct 25 ✅ 8849 ms (24 iters) ✅ 8938 ms (24 iters) ≈ +1% (±10.2%) — / —
finagle-chirper 21 ✅ 5936 ms (33 iters) ✅ 5938 ms (33 iters) ≈ +0% (±24.9%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5534 ms (36 iters) ✅ 5474 ms (36 iters) ≈ -1.1% (±24.5%) ⚠️ W:3 / ⚠️ W:4
fj-kmeans 21 ✅ 2773 ms (67 iters) ✅ 2775 ms (67 iters) ≈ +0.1% (±2.7%) — / —
fj-kmeans 25 ✅ 2803 ms (66 iters) ✅ 2840 ms (66 iters) ≈ +1.3% (±2.6%) — / —
future-genetic 21 ✅ 2090 ms (89 iters) ✅ 2093 ms (89 iters) ≈ +0.1% (±2.7%) — / —
future-genetic 25 ✅ 2086 ms (89 iters) ✅ 2118 ms (87 iters) ≈ +1.5% (±2.4%) — / —
naive-bayes 21 ✅ 1251 ms (136 iters) ✅ 1268 ms (135 iters) ≈ +1.4% (±33.1%) — / —
naive-bayes 25 ✅ 1016 ms (169 iters) ✅ 1028 ms (166 iters) ≈ +1.2% (±31.8%) — / —
reactors 25 ✅ 18334 ms (15 iters) ✅ 18614 ms (15 iters) ≈ +1.5% (±4.6%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 2 / 1 1924 / 1980 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 4 / 2 2127 / 2360 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 4 / 3 8378 / 8654 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8670 / 8433 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1259 / 1251 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 1 / 4 1288 / 1264 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / 4 2942 / 2917 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 1 / 1 2898 / 2871 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 4 / 1 3492 / 3510 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 1 / 3 3496 / 3479 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 2 1858 / 1656 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 1890 / 1893 ✅ / ✅ ✅ / ✅

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from e13ec27 to c111812 Compare July 17, 2026 09:43
@dd-octo-sts

dd-octo-sts Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit c111812)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125236325 Commit: c11181224884586f33d98ba19aa41e6162ecf5f2

✅ Within expected boundaries

No significant runtime deltas (all within run-to-run noise) and no internal-counter outliers.

Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10288 ms (21 iters) ✅ 10150 ms (21 iters) ≈ -1.3% (±11.5%) — / —
akka-uct 25 ✅ 8875 ms (24 iters) ✅ 8975 ms (24 iters) ≈ +1.1% (±10.6%) — / —
finagle-chirper 25 ✅ 5508 ms (36 iters) ✅ 5491 ms (36 iters) ≈ -0.3% (±24.3%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2661 ms (69 iters) ✅ 2713 ms (69 iters) ≈ +2% (±2.7%) — / —
fj-kmeans 25 ✅ 2865 ms (66 iters) ✅ 2841 ms (66 iters) ≈ -0.8% (±2.6%) — / —
future-genetic 21 ✅ 2107 ms (88 iters) ✅ 2126 ms (87 iters) ≈ +0.9% (±2.7%) — / —
future-genetic 25 ✅ 2068 ms (89 iters) ✅ 2110 ms (88 iters) ≈ +2% (±2.6%) — / —
naive-bayes 21 ✅ 1292 ms (132 iters) ✅ 1224 ms (139 iters) ≈ -5.3% (±31.9%) — / —
naive-bayes 25 ✅ 1012 ms (169 iters) ✅ 984 ms (173 iters) ≈ -2.8% (±31.3%) — / —
reactors 21 ✅ 16274 ms (15 iters) ✅ 17426 ms (15 iters) ≈ +7.1% (±7.6%) — / —
reactors 25 ✅ 18845 ms (15 iters) ✅ 18270 ms (15 iters) ≈ -3.1% (±3.8%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 3 / 4 2097 / 1909 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 3 / 1 2062 / 2338 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 1 / 2 8704 / 8842 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 3 / 2 8473 / 8584 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 1 / 1 1246 / 1251 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 1 / 2 1309 / 1292 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 2 / 2 2935 / 2923 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2825 / 2908 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 5 / 4 3535 / 3528 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 4 / 6 3515 / 3473 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 1 / 1 1706 / 1935 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1880 / 1862 ✅ / ✅ ✅ / ✅

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from c111812 to df472c0 Compare July 20, 2026 02:03
@dd-octo-sts

dd-octo-sts Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit df472c0)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125568035 Commit: df472c043ef31dc15b077cc33c5116e3b9f11e23

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +5% (2692→2827 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 25 ✅ 8806 ms (24 iters) ✅ 8924 ms (24 iters) ≈ +1.3% (±11.1%) — / —
finagle-chirper 21 ✅ 6033 ms (33 iters) ✅ 5987 ms (33 iters) ≈ -0.8% (±25.3%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5500 ms (36 iters) ✅ 5495 ms (36 iters) ≈ -0.1% (±24.2%) ⚠️ W:3 / ⚠️ W:4
fj-kmeans 21 ✅ 2692 ms (70 iters) ✅ 2827 ms (66 iters) 🔴 +5% — / —
future-genetic 21 ✅ 2075 ms (89 iters) ✅ 2042 ms (91 iters) ≈ -1.6% (±2.6%) — / —
future-genetic 25 ✅ 2036 ms (91 iters) ✅ 2023 ms (91 iters) ≈ -0.6% (±2.8%) — / —
naive-bayes 21 ✅ 1228 ms (138 iters) ✅ 1283 ms (133 iters) ≈ +4.5% (±33.5%) — / —
reactors 21 ✅ 16202 ms (15 iters) ✅ 16354 ms (15 iters) ≈ +0.9% (±6.6%) — / —
reactors 25 ✅ 18212 ms (15 iters) ✅ 18060 ms (15 iters) ≈ -0.8% (±4.8%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 25 ✅ / ✅ ✅ / ✅ 2 / 5 2176 / 2305 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 1 / 2 8602 / 8811 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8746 / 8149 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ ✅ / 2 1255 / 1277 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 3 / 3 1280 / 1287 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 4 / ✅ 2966 / 2985 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 3 / ✅ 2906 / 2872 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 5 / 3 3486 / 3512 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 2 / 2 3483 / 3521 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 1 / ✅ 1654 / 1756 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 1822 / 1798 ✅ / ✅ ✅ / ✅

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from df472c0 to 46fdadd Compare July 20, 2026 08:03
@dd-octo-sts

dd-octo-sts Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 46fdadd)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125595112 Commit: 46fdadd7a212b7224c08b7dea5a0e2f8dff3166d

✅ Within expected boundaries

No significant runtime deltas (all within run-to-run noise) and no internal-counter outliers.

Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10206 ms (21 iters) ✅ 10428 ms (21 iters) ≈ +2.2% (±11.8%) — / —
akka-uct 25 ✅ 8818 ms (24 iters) ✅ 8800 ms (24 iters) ≈ -0.2% (±9.8%) — / —
finagle-chirper 21 ✅ 6051 ms (33 iters) ✅ 5960 ms (33 iters) ≈ -1.5% (±25%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5475 ms (36 iters) ✅ 5507 ms (36 iters) ≈ +0.6% (±24.1%) ⚠️ W:3 / ⚠️ W:4
fj-kmeans 21 ✅ 2700 ms (70 iters) ✅ 2757 ms (68 iters) ≈ +2.1% (±2.8%) — / —
fj-kmeans 25 ✅ 2839 ms (66 iters) ✅ 2853 ms (66 iters) ≈ +0.5% (±2.5%) — / —
future-genetic 21 ✅ 2113 ms (88 iters) ✅ 2118 ms (88 iters) ≈ +0.2% (±2.7%) — / —
future-genetic 25 ✅ 2026 ms (91 iters) ✅ 2070 ms (90 iters) ≈ +2.2% (±2.7%) — / —
naive-bayes 21 ✅ 1254 ms (137 iters) ✅ 1245 ms (137 iters) ≈ -0.7% (±32.1%) — / —
naive-bayes 25 ✅ 1028 ms (166 iters) ✅ 1020 ms (168 iters) ≈ -0.8% (±31.4%) — / —
reactors 21 ✅ 16534 ms (15 iters) ✅ 16250 ms (15 iters) ≈ -1.7% (±8.7%) — / —
reactors 25 ✅ 18124 ms (15 iters) ✅ 18095 ms (15 iters) ≈ -0.2% (±4.2%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 1 / 5 2079 / 1934 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 2051 / 2211 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 4 / 5 8791 / 8807 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 8270 / 8308 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 2 / 1 1284 / 1300 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 2 / 1 1297 / 1291 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 2 / 1 3010 / 3036 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ ✅ / 2 2860 / 2882 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 2 / 4 3492 / 3544 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 1 / 6 3485 / 3508 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / 2 1656 / 1697 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / 2 1826 / 1779 ✅ / ✅ ✅ / ✅

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from 46fdadd to 658df77 Compare July 20, 2026 14:35
@dd-octo-sts

dd-octo-sts Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 658df77)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125689745 Commit: 658df7762e09e57c0b9c835a39c42495922d564f

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +3.5% (2679→2773 ms)
  • 🔴 future-genetic (JDK 21): runtime +4% (2051→2133 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10260 ms (21 iters) ✅ 10296 ms (21 iters) ≈ +0.4% (±10.9%) — / —
akka-uct 25 ✅ 8840 ms (24 iters) ✅ 8942 ms (24 iters) ≈ +1.2% (±10.6%) — / —
finagle-chirper 21 ✅ 6010 ms (33 iters) ✅ 5968 ms (33 iters) ≈ -0.7% (±25.4%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5480 ms (36 iters) ✅ 5467 ms (36 iters) ≈ -0.2% (±24.6%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2679 ms (69 iters) ✅ 2773 ms (67 iters) 🔴 +3.5% — / —
fj-kmeans 25 ✅ 2769 ms (68 iters) ✅ 2839 ms (66 iters) ≈ +2.5% (±2.8%) — / —
future-genetic 21 ✅ 2051 ms (90 iters) ✅ 2133 ms (87 iters) 🔴 +4% — / —
future-genetic 25 ✅ 2047 ms (91 iters) ✅ 2037 ms (91 iters) ≈ -0.5% (±2.5%) — / —
naive-bayes 21 ✅ 1254 ms (136 iters) ✅ 1314 ms (131 iters) ≈ +4.8% (±33.9%) — / —
naive-bayes 25 ✅ 1015 ms (169 iters) ✅ 983 ms (173 iters) ≈ -3.2% (±31%) — / —
reactors 21 ✅ 16594 ms (15 iters) ✅ 16990 ms (15 iters) ≈ +2.4% (±9.1%) — / —
reactors 25 ✅ 18557 ms (15 iters) ✅ 18660 ms (15 iters) ≈ +0.6% (±4.3%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 2 / 1 1856 / 2016 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 2 / 1 2373 / 2253 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 5 / 7 8704 / 8448 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 2 / 2 8503 / 8277 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 1 / 2 1260 / 1244 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ 4 / 1 1282 / 1284 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 1 / ✅ 2984 / 2927 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 1 / 3 2889 / 2841 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 4 / 3 3476 / 3519 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 2 / 6 3442 / 3468 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ 2 / 1 1680 / 1865 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ ✅ / 1 1868 / 1860 ✅ / ✅ ✅ / ✅

Copilot AI review requested due to automatic review settings July 21, 2026 07:37
@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from 658df77 to 746edc2 Compare July 21, 2026 07:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds first-class TaskBlock emission for common JVM blocking states (park, Object.wait, monitor contention), and reworks wall-clock precheck/suppression to use lifecycle-owned blocked intervals (instead of the prior once-per-run suppression model) while preserving context-window sampling semantics.

Changes:

  • Introduces a new datadog.TaskBlock JFR event and a public paired Java API (beginTaskBlock / endTaskBlock) plus internal park/monitor producers.
  • Implements native lifecycle ownership + unfiltered-thread registry tracking and a bounded candidate backfill strategy to prevent O(N) wall-tick regressions.
  • Adds broad Java + C++ test coverage (including restart/pre-existing-thread scenarios, JVMTI-delegated variants, and benchmark hooks).

Reviewed changes

Copilot reviewed 59 out of 59 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/WallclockMitigationsCombinedTest.java Updates combined mitigation test expectations to ensure context-scoped threads remain sampled.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/UnfilteredWallPrecheckTest.java New integration tests for owned-block suppression behavior when filter= samples all threads.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/UnfilteredWallPrecheckRestartTest.java New tests ensuring unfiltered tracking/registry state is correctly reset across recording restarts.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/TaskBlockAssertions.java New shared assertions for the datadog.TaskBlock event contract.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/PrecheckTest.java Refactors precheck tests to the new lifecycle-owned suppression model and unowned blocked weighting counters.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/PrecheckEfficiencyTest.java Updates efficiency tests to match lifecycle ownership semantics and explicit unfiltered sampling configuration.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/ParkTaskBlockTest.java New tests verifying TaskBlock emission from Java-owned park hooks and suppression counters.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/MonitorTaskBlockTest.java New tests verifying TaskBlock emission from JVMTI monitor callbacks and restart recovery.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JvmtiBasedUnfilteredWallPrecheckTest.java New subclass to exercise unfiltered precheck via delegated JVMTI stack collection.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JvmtiBasedPrecheckTest.java Updates command strings to use explicit filter= under new behavior.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JvmtiBasedParkTaskBlockTest.java New JVMTI-delegated variant of park TaskBlock tests.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JvmtiBasedMonitorTaskBlockTest.java New JVMTI-delegated variant of monitor TaskBlock tests.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JavaProfilerTaskBlockPreExistingThreadTest.java New test covering TaskBlock TLS initialization for threads created before profiler startup.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JavaProfilerTaskBlockDisabledTest.java New test verifying TaskBlock API stays inactive outside any sampling scope.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/JavaProfilerTaskBlockApiTest.java New end-to-end tests for the paired TaskBlock API semantics and rejection rules.
ddprof-test/src/test/java/com/datadoghq/profiler/wallclock/J9WallClockPrecheckCapabilityTest.java New test ensuring J9 wall sampling doesn’t activate unfiltered precheck tracking.
ddprof-test/src/test/java/com/datadoghq/profiler/JavaProfilerTest.java Adds child-JVM agentpath initialization tests and pre-existing monitor callback coverage.
ddprof-test/src/test/java/com/datadoghq/profiler/JavaProfilerApiSurfaceTest.java Updates API-surface assertions: TaskBlock API public, internal hooks stay non-public, monitor delegation APIs are public.
ddprof-test/src/test/java/com/datadoghq/profiler/ExternalLauncher.java Extends launcher scenarios to cover agent compatibility, delegation conflicts, and pre-existing monitor callbacks.
ddprof-test/src/test/java/com/datadoghq/profiler/context/OtelContextStorageModeTest.java Ensures trace context is cleared during cleanup alongside thread context reset.
ddprof-test/src/test/java/com/datadoghq/profiler/context/AllNativeContextTest.java Ensures trace context is cleared during cleanup alongside thread context reset.
ddprof-test/src/test/java/com/datadoghq/profiler/AbstractProfilerTest.java Adds beforeProfilerStart() hook and improves teardown to clear trace context reliably.
ddprof-stresstest/src/jmh/java/com/datadoghq/profiler/WallClockPrecheckBenchmarkHooks.java New helper exposing package-scoped owned-block hooks to JMH benchmarks.
ddprof-stresstest/src/jmh/java/com/datadoghq/profiler/stresstest/scenarios/throughput/WallClockPrecheckOverheadBenchmark.java New benchmark measuring wall-clock timer overhead under growing owned-block populations.
ddprof-lib/src/test/cpp/wallprecheck_args_ut.cpp Adds capability + filter parsing tests relevant to unfiltered tracking (filter omitted vs filter=).
ddprof-lib/src/test/cpp/wallClockCounters_ut.cpp Updates counter tests to the renamed owned-block suppression metric.
ddprof-lib/src/test/cpp/wallClockCandidateSelector_ut.cpp New unit tests for randomized bounded candidate selection/backfill.
ddprof-lib/src/test/cpp/taskBlockRecorder_ut.cpp New unit tests for TaskBlock eligibility, rotation gating, and error accounting.
ddprof-lib/src/test/cpp/park_state_ut.cpp Expands state-machine tests for park and monitor snapshot/exit behavior and owned-block transitions.
ddprof-lib/src/test/cpp/jvmSupport_ut.cpp New unit tests for platform-thread classification (virtual thread rejection) across JNI versions.
ddprof-lib/src/test/cpp/frame_ut.cpp Adds unit test for safe JVMTI->ASGCT frame copying with overlapping buffers.
ddprof-lib/src/main/java/com/datadoghq/profiler/JavaProfiler.java Adds monitor-delegation-aware getInstance, TaskBlock public API, and thread-parameterized native hooks.
ddprof-lib/src/main/cpp/wallClockCounters.h Renames and redefines wall-clock suppression counter to owned-block semantics.
ddprof-lib/src/main/cpp/wallClockCandidateSelector.h New header implementing bounded randomized candidate visitation/backfill.
ddprof-lib/src/main/cpp/wallClock.h Integrates candidate selector into wall timer loop and adds engine capability declaration.
ddprof-lib/src/main/cpp/wallClock.cpp Reworks precheck logic to rely on owned-block registry suppression and bounded slot lookup backfill.
ddprof-lib/src/main/cpp/vmEntry.h Adds monitor delegation ownership model and initialization result enum.
ddprof-lib/src/main/cpp/vmEntry.cpp Implements JVMTI monitor callbacks producing TaskBlocks, plus one-time bridge ownership negotiation.
ddprof-lib/src/main/cpp/threadLocalData.h Adds per-thread state for park/monitor/taskblock lifecycles and snapshotting.
ddprof-lib/src/main/cpp/threadFilter.h Adds unfiltered registry tracking, TID index, recording epochs, and owned-block suppression candidate checks.
ddprof-lib/src/main/cpp/threadFilter.cpp Implements registry activation, per-recording epoch publication, lock-protected indexing, and slot lookup APIs.
ddprof-lib/src/main/cpp/taskBlockRecorder.h New TaskBlock recorder API + eligibility helpers + rotation-safe activity RAII.
ddprof-lib/src/main/cpp/taskBlockRecorder.cpp Implements TaskBlock duration threshold and recording/cleanup at exit paths.
ddprof-lib/src/main/cpp/profiler.h Adds TaskBlock recording API, rotation gating fields, and bootstrap registration hook.
ddprof-lib/src/main/cpp/profiler.cpp Implements TaskBlock recording through JVMTI stack capture, rotation gating, registry bootstrap for existing threads, and start/stop integration.
ddprof-lib/src/main/cpp/jvmThread.h Adds supportsNativeThreadIdLookup() used for registry bootstrap logic.
ddprof-lib/src/main/cpp/jvmThread.cpp Implements native-thread-id lookup capability check.
ddprof-lib/src/main/cpp/jvmSupport.h Adds platform-thread classification API.
ddprof-lib/src/main/cpp/jvmSupport.cpp Implements platform-thread classification (virtual-thread rejection) using JNI version + IsVirtualThread.
ddprof-lib/src/main/cpp/jfrMetadata.h Adds new JFR type id for datadog.TaskBlock.
ddprof-lib/src/main/cpp/jfrMetadata.cpp Defines datadog.TaskBlock metadata schema.
ddprof-lib/src/main/cpp/javaApi.cpp Wires new JNI entrypoints for TaskBlock API and updates park/block hooks to be thread-aware and emit TaskBlocks.
ddprof-lib/src/main/cpp/frames.h Adds copyJvmtiFrames() helper to safely convert overlapping JVMTI frame buffers.
ddprof-lib/src/main/cpp/flightRecorder.h Adds TaskBlock recording API to the JFR writer interface.
ddprof-lib/src/main/cpp/flightRecorder.cpp Implements TaskBlock event serialization and a rotation-safe record path.
ddprof-lib/src/main/cpp/event.h Adds TaskBlockEvent struct and updates wall-clock epoch suppressed counter naming.
ddprof-lib/src/main/cpp/engine.h Adds wall-engine capability hook supportsUnfilteredWallPrecheck().
ddprof-lib/src/main/cpp/counters.h Adds new counters for registry/backfill and TaskBlock lifecycle/accounting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ddprof-lib/src/main/java/com/datadoghq/profiler/JavaProfiler.java
Comment thread ddprof-lib/src/main/cpp/wallClockCandidateSelector.h
Comment thread ddprof-lib/src/test/cpp/wallClockCandidateSelector_ut.cpp
Comment thread ddprof-lib/src/main/cpp/frames.h Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (commit 746edc2)

Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/125879034 Commit: 746edc2a235e9d064e56b38261b8e7b630f9c846

⚠️ Significant outliers

  • 🔴 fj-kmeans (JDK 21): runtime +4.4% (2634→2749 ms)
  • 🟢 reactors (JDK 21): runtime -10.8% (17619→15712 ms)
Runtime details (per benchmark × JDK)
Benchmark JDK Latest Dev Δ (dev vs latest) Issues L/D
akka-uct 21 ✅ 10315 ms (21 iters) ✅ 10281 ms (21 iters) ≈ -0.3% (±11.1%) — / —
akka-uct 25 ✅ 8869 ms (24 iters) ✅ 8962 ms (24 iters) ≈ +1% (±10%) — / —
finagle-chirper 21 ✅ 6014 ms (33 iters) ✅ 5992 ms (33 iters) ≈ -0.4% (±25.2%) ⚠️ W:3 / ⚠️ W:3
finagle-chirper 25 ✅ 5449 ms (36 iters) ✅ 5424 ms (36 iters) ≈ -0.5% (±24%) ⚠️ W:3 / ⚠️ W:3
fj-kmeans 21 ✅ 2634 ms (72 iters) ✅ 2749 ms (68 iters) 🔴 +4.4% — / —
fj-kmeans 25 ✅ 2757 ms (68 iters) ✅ 2825 ms (66 iters) ≈ +2.5% (±2.8%) — / —
future-genetic 21 ✅ 2097 ms (88 iters) ✅ 2137 ms (87 iters) ≈ +1.9% (±2.8%) — / —
future-genetic 25 ✅ 2067 ms (89 iters) ✅ 2031 ms (91 iters) ≈ -1.7% (±2.7%) — / —
naive-bayes 21 ✅ 1253 ms (136 iters) ✅ 1253 ms (137 iters) ≈ 0% (±32.9%) — / —
naive-bayes 25 ✅ 1008 ms (169 iters) ✅ 1002 ms (170 iters) ≈ -0.6% (±31.9%) — / —
reactors 21 ✅ 17619 ms (15 iters) ✅ 15712 ms (16 iters) 🟢 -10.8% — / —
reactors 25 ✅ 18550 ms (15 iters) ✅ 18609 ms (15 iters) ≈ +0.3% (±4%) — / —
Internal counter details (ddprof)

ddprof internal counters, latest / dev (✅ = 0, · = unavailable):

Benchmark JDK Dropped rec Dropped jvmti Dropped trace Skipped WC AGCT fail Unwind fail
akka-uct 21 ✅ / ✅ ✅ / ✅ 3 / ✅ 1987 / 1971 ✅ / ✅ ✅ / ✅
akka-uct 25 ✅ / ✅ ✅ / ✅ 3 / 2 2310 / 2435 ✅ / ✅ ✅ / ✅
finagle-chirper 21 ✅ / ✅ ✅ / ✅ 1 / 3 8921 / 8512 ✅ / ✅ ✅ / ✅
finagle-chirper 25 ✅ / ✅ ✅ / ✅ 2 / 1 8447 / 8030 ✅ / ✅ ✅ / ✅
fj-kmeans 21 ✅ / ✅ ✅ / ✅ 4 / 1 1274 / 1266 ✅ / ✅ ✅ / ✅
fj-kmeans 25 ✅ / ✅ ✅ / ✅ ✅ / 1 1280 / 1270 ✅ / ✅ ✅ / ✅
future-genetic 21 ✅ / ✅ ✅ / ✅ 4 / 1 2912 / 3053 ✅ / ✅ ✅ / ✅
future-genetic 25 ✅ / ✅ ✅ / ✅ 2 / 1 2767 / 2834 ✅ / ✅ ✅ / ✅
naive-bayes 21 ✅ / ✅ ✅ / ✅ 2 / 1 3517 / 3509 ✅ / ✅ ✅ / ✅
naive-bayes 25 ✅ / ✅ ✅ / ✅ 6 / 6 3483 / 3497 ✅ / ✅ ✅ / ✅
reactors 21 ✅ / ✅ ✅ / ✅ ✅ / ✅ 1802 / 1666 ✅ / ✅ ✅ / ✅
reactors 25 ✅ / ✅ ✅ / ✅ 1 / ✅ 1883 / 1920 ✅ / ✅ ✅ / ✅

@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from 746edc2 to 1db6b94 Compare July 21, 2026 14:29
@kaahos
kaahos force-pushed the paul.fournillon/taskblock-jvm-producers branch from 1db6b94 to e9b2308 Compare July 21, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants