Skip to content

test(dflash): add exact prefill differential tracing - #592

Draft
cheese-cakee wants to merge 9 commits into
Luce-Org:mainfrom
cheese-cakee:codex/ds4-exact-diff-harness
Draft

test(dflash): add exact prefill differential tracing#592
cheese-cakee wants to merge 9 commits into
Luce-Org:mainfrom
cheese-cakee:codex/ds4-exact-diff-harness

Conversation

@cheese-cakee

@cheese-cakee cheese-cakee commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a default-off production diagnostic for exact differential validation of DeepSeek4 prefill widths.

  • emits authenticated JSONL traces for routing, HC state, raw and compressed KV state, captures, logits, continuations, reset, and snapshot lifecycle;
  • compares q2, q3, and q4 against a q1 oracle with fixed fail-closed tolerances and exact lifecycle checks;
  • runs an eight-trace reset/snapshot matrix and binds every trace to one binary, revision, model set, prompt, and request configuration;
  • keeps synchronous trace readbacks disabled unless DFLASH_DS4_EXACT_TRACE_PATH is set.

This is separate from the performance change so the diagnostic harness and its product documentation can be reviewed independently.

Dependency

The q2 through q4 matrix requires the exact-band implementation from draft PR #590, or an equivalent implementation. Current main does not provide that behavior, so the standalone end-to-end matrix is expected to fail closed until the dependency is integrated.

Scope

The PR contains 11 product, harness, schema, and unit-test files. It contains no local audit, state, handoff, evidence, .lucebox, or personal Markdown files.

Verification

Exact head: db0c609d603824e39e54b37809f1f0404e171cbb

  • focused Python comparator tests: 39 passed;
  • Ruff lint and format checks: passed;
  • Python compileall: passed;
  • git diff --check: passed;
  • fresh independent final-diff review: no actionable offline blocker.

An exact-head C++ rebuild was attempted after rebasing onto b8c3a0d, but exceeded the five-minute bounded run and is incomplete. It is not reported as a pass. Earlier CUDA build and CTest results were invalidated by the new base and are not exact-head evidence.

Current limitations

  • No compatible-model production trace matrix has run on this head.
  • No HIP/gfx1151 target validation has run.
  • No performance claim is made; tracing intentionally adds synchronous diagnostic readbacks.
  • This PR is not merge-ready until the exact-band dependency is integrated and the C++ plus real model/target validation is completed.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant