Profiler's fuzz tests store corpus files under wrong directories#656
Open
zhengyu123 wants to merge 3 commits into
Open
Profiler's fuzz tests store corpus files under wrong directories#656zhengyu123 wants to merge 3 commits into
zhengyu123 wants to merge 3 commits into
Conversation
Contributor
Benchmark Results (commit 630a378)Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124392220 Commit:
|
| Benchmark | JDK | Latest | Dev | Δ (dev vs latest) | Issues L/D |
|---|---|---|---|---|---|
| akka-uct | 25 | ✅ 8903 ms (24 iters) | ✅ 8848 ms (24 iters) | ≈ -0.6% (±10.2%) | — / — |
| finagle-chirper | 21 | ✅ 5973 ms (33 iters) | ✅ 5965 ms (33 iters) | ≈ -0.1% (±24.3%) | |
| finagle-chirper | 25 | ✅ 5510 ms (36 iters) | ✅ 5519 ms (36 iters) | ≈ +0.2% (±24.6%) | |
| fj-kmeans | 21 | ✅ 2771 ms (68 iters) | ✅ 2662 ms (69 iters) | 🟢 -3.9% | — / — |
| fj-kmeans | 25 | ✅ 2771 ms (68 iters) | ✅ 2824 ms (66 iters) | ≈ +1.9% (±2.8%) | — / — |
| future-genetic | 21 | ✅ 2072 ms (89 iters) | ✅ 2132 ms (87 iters) | 🔴 +2.9% | — / — |
| future-genetic | 25 | ✅ 1994 ms (93 iters) | ✅ 2117 ms (88 iters) | 🔴 +6.2% | — / — |
| naive-bayes | 21 | ✅ 1281 ms (134 iters) | ✅ 1295 ms (133 iters) | ≈ +1.1% (±32.5%) | — / — |
| naive-bayes | 25 | ✅ 964 ms (177 iters) | ✅ 1009 ms (169 iters) | ≈ +4.7% (±32.6%) | — / — |
| reactors | 21 | ✅ 16279 ms (15 iters) | ✅ 16615 ms (15 iters) | ≈ +2.1% (±8.5%) | — / — |
| reactors | 25 | ✅ 18517 ms (15 iters) | ✅ 18578 ms (15 iters) | ≈ +0.3% (±3.7%) | — / — |
Internal counter details (ddprof)
ddprof internal counters, latest / dev (✅ = 0, · = unavailable):
| Benchmark | JDK | Dropped rec | Dropped jvmti | Dropped trace | Skipped WC | AGCT fail | Unwind fail |
|---|---|---|---|---|---|---|---|
| akka-uct | 21 | ✅ / ✅ | ✅ / ✅ | ✅ / ✅ | ✅ / ✅ | ✅ / ✅ | ✅ / ✅ |
| akka-uct | 25 | ✅ / ✅ | ✅ / ✅ | 1 / 1 | 2328 / 2246 | ✅ / ✅ | ✅ / ✅ |
| finagle-chirper | 21 | ✅ / ✅ | ✅ / ✅ | 2 / 3 | 8740 / 8675 | ✅ / ✅ | ✅ / ✅ |
| finagle-chirper | 25 | ✅ / ✅ | ✅ / ✅ | ✅ / 2 | 8359 / 8233 | ✅ / ✅ | ✅ / ✅ |
| fj-kmeans | 21 | ✅ / ✅ | ✅ / ✅ | ✅ / 2 | 1260 / 1242 | ✅ / ✅ | ✅ / ✅ |
| fj-kmeans | 25 | ✅ / ✅ | ✅ / ✅ | 1 / 1 | 1303 / 1267 | ✅ / ✅ | ✅ / ✅ |
| future-genetic | 21 | ✅ / ✅ | ✅ / ✅ | ✅ / 1 | 2925 / 2934 | ✅ / ✅ | ✅ / ✅ |
| future-genetic | 25 | ✅ / ✅ | ✅ / ✅ | 2 / 2 | 2911 / 2877 | ✅ / ✅ | ✅ / ✅ |
| naive-bayes | 21 | ✅ / ✅ | ✅ / ✅ | 1 / 4 | 3533 / 3525 | ✅ / ✅ | ✅ / ✅ |
| naive-bayes | 25 | ✅ / ✅ | ✅ / ✅ | 3 / 2 | 3480 / 3456 | ✅ / ✅ | ✅ / ✅ |
| reactors | 21 | ✅ / ✅ | ✅ / ✅ | 1 / 2 | 1705 / 1715 | ✅ / ✅ | ✅ / ✅ |
| reactors | 25 | ✅ / ✅ | ✅ / ✅ | ✅ / 1 | 1851 / 1839 | ✅ / ✅ | ✅ / ✅ |
Contributor
CI Test ResultsRun: #29320660052 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Failed Testsglibc-amd64/debug / 11-j9Job: View logs No detailed failure information available. Check the job logs. Summary: Total: 32 | Passed: 31 | Failed: 1 Updated: 2026-07-14 09:26:40 UTC |
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes fuzzing seed-corpus discovery by aligning committed corpus directories with the fuzz plugin’s derived target names (stripping the fuzz_ prefix), so fuzzers start with the intended seeds instead of an empty corpus.
Changes:
- Updates fuzzing docs/comments to reference
corpus/<targetName>/(withoutfuzz_). - Adds/relocates seed corpus entries under the corrected per-target directories (e.g.,
corpus/threadLocal/,corpus/arguments/, etc.). - Adds new lifecycle/bit-pattern seeds for the
threadLocalfuzz target.
Reviewed changes
Copilot reviewed 2 out of 24 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| ddprof-lib/src/test/fuzz/README.md | Documentation updated to reflect corpus/<target>/ directory naming. |
| ddprof-lib/src/test/fuzz/fuzz_arguments.cpp | Comment updated to point to the corrected corpus/arguments/ seed directory. |
| ddprof-lib/src/test/fuzz/corpus/threadLocal/tracked_overwrite_padding | Seed corpus entry for threadLocal target. |
| ddprof-lib/src/test/fuzz/corpus/threadLocal/tracked_nullptr_recreate | Seed corpus entry for threadLocal target. |
| ddprof-lib/src/test/fuzz/corpus/threadLocal/tracked_lifecycle | Seed corpus entry for threadLocal target. |
| ddprof-lib/src/test/fuzz/corpus/threadLocal/intptr_roundtrip | Seed corpus entry for threadLocal target. |
| ddprof-lib/src/test/fuzz/corpus/threadLocal/double_truncated_payload | Seed corpus entry for threadLocal target. |
| ddprof-lib/src/test/fuzz/corpus/threadLocal/double_special_values | Seed corpus entry for threadLocal target. |
| ddprof-lib/src/test/fuzz/corpus/stringDictionary/basic_rotation | Seed corpus entry for stringDictionary target. |
| ddprof-lib/src/test/fuzz/corpus/elf/bug3_buildid_note_overflow | Seed corpus entry for elf target. |
| ddprof-lib/src/test/fuzz/corpus/elf/bug2_symtab_size_oob | Seed corpus entry for elf target. |
| ddprof-lib/src/test/fuzz/corpus/elf/bug1_section_header_oob_2 | Seed corpus entry for elf target. |
| ddprof-lib/src/test/fuzz/corpus/elf/bug1_section_header_oob_1 | Seed corpus entry for elf target. |
| ddprof-lib/src/test/fuzz/corpus/dwarf/minimal_eh_frame_hdr | Seed corpus entry for dwarf target. |
| ddprof-lib/src/test/fuzz/corpus/callTraceStorage/seed0 | Seed corpus entry for callTraceStorage target. |
| ddprof-lib/src/test/fuzz/corpus/buffer/varint_max | Seed corpus entry for buffer target. |
| ddprof-lib/src/test/fuzz/corpus/buffer/utf8_string | Seed corpus entry for buffer target. |
| ddprof-lib/src/test/fuzz/corpus/buffer/put8_sequence | Seed corpus entry for buffer target. |
| ddprof-lib/src/test/fuzz/corpus/arguments/stop | Seed corpus entry for arguments target. |
| ddprof-lib/src/test/fuzz/corpus/arguments/start_wall_file | Seed corpus entry for arguments target. |
| ddprof-lib/src/test/fuzz/corpus/arguments/start_cpu | Seed corpus entry for arguments target. |
| ddprof-lib/src/test/fuzz/corpus/arguments/start_alloc | Seed corpus entry for arguments target. |
| ddprof-lib/src/test/fuzz/corpus/arguments/cstack_dwarf | Seed corpus entry for arguments target. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
Reliability & Chaos Results✅ All reliability & chaos checks passed Pipeline: https://gitlab.ddbuild.io/DataDog/java-profiler/-/pipelines/124403086 |
This comment has been minimized.
This comment has been minimized.
Contributor
Benchmark Results (commit 53f4cc2)Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124403098 Commit:
|
| Benchmark | JDK | Latest | Dev | Δ (dev vs latest) | Issues L/D |
|---|---|---|---|---|---|
| akka-uct | 21 | ✅ 10371 ms (21 iters) | ✅ 10197 ms (21 iters) | ≈ -1.7% (±11.2%) | — / — |
| akka-uct | 25 | ✅ 8776 ms (24 iters) | ✅ 8946 ms (24 iters) | ≈ +1.9% (±11.1%) | — / — |
| finagle-chirper | 21 | ✅ 5958 ms (33 iters) | ✅ 6039 ms (33 iters) | ≈ +1.4% (±25.5%) | |
| fj-kmeans | 21 | ✅ 2670 ms (71 iters) | ✅ 2762 ms (68 iters) | 🔴 +3.4% | — / — |
| future-genetic | 25 | ✅ 2140 ms (87 iters) | ✅ 2084 ms (89 iters) | 🟢 -2.6% | — / — |
| naive-bayes | 21 | ✅ 1239 ms (138 iters) | ✅ 1295 ms (132 iters) | ≈ +4.5% (±33.5%) | — / — |
| naive-bayes | 25 | ✅ 1019 ms (168 iters) | ✅ 1010 ms (170 iters) | ≈ -0.9% (±31.4%) | — / — |
| reactors | 21 | ✅ 16396 ms (15 iters) | ✅ 16350 ms (16 iters) | ≈ -0.3% (±8.2%) | — / — |
| reactors | 25 | ✅ 18761 ms (15 iters) | ✅ 18273 ms (15 iters) | ≈ -2.6% (±4.8%) | — / — |
Internal counter details (ddprof)
ddprof internal counters, latest / dev (✅ = 0, · = unavailable):
| Benchmark | JDK | Dropped rec | Dropped jvmti | Dropped trace | Skipped WC | AGCT fail | Unwind fail |
|---|---|---|---|---|---|---|---|
| akka-uct | 21 | ✅ / ✅ | ✅ / ✅ | 4 / 3 | 1881 / 1912 | ✅ / ✅ | ✅ / ✅ |
| akka-uct | 25 | ✅ / ✅ | ✅ / ✅ | 6 / 1 | 2170 / 2389 | ✅ / ✅ | ✅ / ✅ |
| finagle-chirper | 21 | ✅ / ✅ | ✅ / ✅ | 2 / 1 | 8265 / 8595 | ✅ / ✅ | ✅ / ✅ |
| fj-kmeans | 21 | ✅ / ✅ | ✅ / ✅ | 1 / 2 | 1305 / 1274 | ✅ / ✅ | ✅ / ✅ |
| future-genetic | 21 | · / ✅ | · / ✅ | · / ✅ | · / ✅ | · / ✅ | · / ✅ |
| future-genetic | 25 | ✅ / ✅ | ✅ / ✅ | 5 / ✅ | 2888 / 2868 | ✅ / ✅ | ✅ / ✅ |
| naive-bayes | 21 | ✅ / ✅ | ✅ / ✅ | 4 / ✅ | 3532 / 3534 | ✅ / ✅ | ✅ / ✅ |
| naive-bayes | 25 | ✅ / ✅ | ✅ / ✅ | 5 / 3 | 3491 / 3460 | ✅ / ✅ | ✅ / ✅ |
| reactors | 21 | ✅ / ✅ | ✅ / ✅ | ✅ / 2 | 1634 / 1682 | ✅ / ✅ | ✅ / ✅ |
| reactors | 25 | ✅ / ✅ | ✅ / ✅ | 2 / 1 | 1936 / 1824 | ✅ / ✅ | ✅ / ✅ |
Contributor
Benchmark Results (commit 4124e5f)Pipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/124487523 Commit:
|
| Benchmark | JDK | Latest | Dev | Δ (dev vs latest) | Issues L/D |
|---|---|---|---|---|---|
| akka-uct | 21 | ✅ 10124 ms (21 iters) | ✅ 10228 ms (21 iters) | ≈ +1% (±11.3%) | — / — |
| akka-uct | 25 | ✅ 9002 ms (24 iters) | ✅ 8891 ms (24 iters) | ≈ -1.2% (±9.8%) | |
| finagle-chirper | 21 | ✅ 5982 ms (33 iters) | ✅ 6019 ms (33 iters) | ≈ +0.6% (±25.3%) | |
| finagle-chirper | 25 | ✅ 5476 ms (36 iters) | ✅ 5461 ms (36 iters) | ≈ -0.3% (±24.6%) | |
| fj-kmeans | 21 | ✅ 2673 ms (71 iters) | ✅ 2821 ms (66 iters) | 🔴 +5.5% | — / — |
| fj-kmeans | 25 | ✅ 2822 ms (66 iters) | ✅ 2849 ms (66 iters) | ≈ +1% (±2.6%) | — / — |
| future-genetic | 21 | ✅ 2109 ms (88 iters) | ✅ 2084 ms (89 iters) | ≈ -1.2% (±2.8%) | — / — |
| future-genetic | 25 | ✅ 2043 ms (91 iters) | ✅ 2141 ms (87 iters) | 🔴 +4.8% | — / — |
| naive-bayes | 21 | ✅ 1277 ms (134 iters) | ✅ 1326 ms (130 iters) | ≈ +3.8% (±32.7%) | — / — |
| naive-bayes | 25 | ✅ 995 ms (171 iters) | ✅ 957 ms (178 iters) | ≈ -3.8% (±31.2%) | — / — |
| reactors | 21 | ✅ 16225 ms (15 iters) | ✅ 15913 ms (15 iters) | ≈ -1.9% (±7.7%) | — / — |
| reactors | 25 | ✅ 18711 ms (15 iters) | ✅ 18785 ms (15 iters) | ≈ +0.4% (±4.8%) | — / — |
Internal counter details (ddprof)
ddprof internal counters, latest / dev (✅ = 0, · = unavailable):
| Benchmark | JDK | Dropped rec | Dropped jvmti | Dropped trace | Skipped WC | AGCT fail | Unwind fail |
|---|---|---|---|---|---|---|---|
| akka-uct | 21 | ✅ / ✅ | ✅ / ✅ | 3 / ✅ | 1927 / 1969 | ✅ / ✅ | ✅ / ✅ |
| akka-uct | 25 | ✅ / ✅ | ✅ / ✅ | 1 / ✅ | 2335 / 2061 | ✅ / ✅ | ✅ / ✅ |
| finagle-chirper | 21 | ✅ / ✅ | ✅ / ✅ | 1 / 6 | 8161 / 9080 | ✅ / ✅ | ✅ / ✅ |
| finagle-chirper | 25 | ✅ / ✅ | ✅ / ✅ | 3 / ✅ | 8203 / 8159 | ✅ / ✅ | ✅ / ✅ |
| fj-kmeans | 21 | ✅ / ✅ | ✅ / ✅ | 4 / 3 | 1291 / 1254 | ✅ / ✅ | ✅ / ✅ |
| fj-kmeans | 25 | ✅ / ✅ | ✅ / ✅ | ✅ / 2 | 1269 / 1279 | ✅ / ✅ | ✅ / ✅ |
| future-genetic | 21 | ✅ / ✅ | ✅ / ✅ | ✅ / 1 | 2948 / 3020 | ✅ / ✅ | ✅ / ✅ |
| future-genetic | 25 | ✅ / ✅ | ✅ / ✅ | ✅ / 1 | 2836 / 2956 | ✅ / ✅ | ✅ / ✅ |
| naive-bayes | 21 | ✅ / ✅ | ✅ / ✅ | 2 / 3 | 3507 / 3529 | ✅ / ✅ | ✅ / ✅ |
| naive-bayes | 25 | ✅ / ✅ | ✅ / ✅ | 9 / 4 | 3467 / 3508 | ✅ / ✅ | ✅ / ✅ |
| reactors | 21 | ✅ / ✅ | ✅ / ✅ | ✅ / 1 | 1715 / 1506 | ✅ / ✅ | ✅ / ✅ |
| reactors | 25 | ✅ / ✅ | ✅ / ✅ | ✅ / 1 | 1945 / 1908 | ✅ / ✅ | ✅ / ✅ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?:
The fuzz plugin strips the fuzz_ prefix when deriving fuzzName and passes File(corpusBaseDir, fuzzName) to libFuzzer, so fuzz_threadLocal.cpp runs with ddprof-lib/src/test/fuzz/corpus/threadLocal, not corpus/fuzz_threadLocal (see FuzzTargetsPlugin.kt lines 114-120 and 166-168). With the seeds committed and documented under corpus/fuzz_threadLocal, this new target silently starts from an empty corpus and never exercises the intended lifecycle/bit-pattern seeds unless the user manually points libFuzzer at the other directory.
This same mismatch exists for every other existing target already in the repo — corpus/fuzz_arguments/, fuzz_buffer/, fuzz_callTraceStorage/, fuzz_dwarf/, fuzz_elf/, and fuzz_stringDictionary/
Motivation:
Fix bug.
Additional Notes:
How to test the change?:
For Datadog employees:
credentials of any kind, I've requested a security review (run the
dd:platform-security-reviewskill, or file a request via the PSEC review form).
bewairealso runs automatically on every PR.Unsure? Have a question? Request a review!