Skip to content

Deep-search escalation on a caller-set targetHits (+ the evidence, honestly stated) - #282

Open
ms609 wants to merge 17 commits into
cpp-searchfrom
claude/reach-escalation-gate
Open

Deep-search escalation on a caller-set targetHits (+ the evidence, honestly stated)#282
ms609 wants to merge 17 commits into
cpp-searchfrom
claude/reach-escalation-gate

Conversation

@ms609

@ms609 ms609 commented Aug 3, 2026

Copy link
Copy Markdown
Owner

What this is

An opt-in reach lever for MaximizeParsimony(): when the caller raises targetHits to at
least twice its size-scaled default, at effort rung 3 (thorough) or above, the per-replicate
perturbation itself deepens — more drift, a deep reweighting kick, a post-ratchet sectorial pass,
intraFuse, and internal near-optimal pool retention.

Read the evidence before the code

The honest summary is narrower than the feature sounds, and the review that matters is of that
claim, not of the diff:

  • Across two A/Bs (125 + 55 cells, 25 and 11 matrices), the effect is one matrix:
    project4284 at 4062 tips, 10 win / 0 loss over 10 seeds.
  • project2771 pools to 4 win / 2 loss / 4 tie — noise.
  • Nine other large/xlarge matrices: 0 win / 0 loss / 45 tie, at ~3.5× the wall.
  • The margin is not an effect size. Re-running those cells moved scores several steps in both
    directions: a deadline-truncated run completing zero replicates stops wherever the clock lands.
    The win/loss count replicates; the gap does not.
  • project4284 won having completed zero replicates, so the mechanism is depth within one
    replicate — the opposite lever to restart volume, and a reason not to expect the two to compose.

So this is deliberately off by default and gated, not a new default. dev/benchmarks/reach_escalation_FINDINGS.md
carries the full study including the parts that did not survive.

Two corrections carried in this branch

Both were mine, and both were caught by fixing a bug in my own analyser:

  1. The deadline detector tested the wrong threshold. The engine stops at
    maxSeconds × (1 − enumTimeFraction) — 0.9×, not 1.0× — so a 0.95 × cap test reported
    1 of 110 deadline-bound cells when the true count was 59. Re-derived: 85 cells converged
    in both arms, 36/40 stopped at the deadline, 4 are asymmetric (all tied at 563). Both arms at
    one shared deadline is still a valid equal-wall comparison, so no conclusion moves — but the
    earlier "no cell hit its wall cap" claim was false and is gone.
  2. A 3× score discrepancy against historical CSVs was not the objective, the matrix, or the
    budget.
    It is Wagner addition quality, since fixed: bare AdditionTree, same data and seed,
    no search — March 1590 vs current 409. Recorded without attributing it to a single
    commit, because nothing was bisected.

The trigger, and why there is no effort door

ea74613c gated on targetHits / defaultHits >= 2 alone. Upstream's effort ladder now satisfies
that by itself — .RungSpec()'s hitMultiplier doubles targetHits at rung 5 precisely when
the user did not set it
— so on any dataset ≥120 tips, effort = 1 fired the bundle on a
mechanical artefact, at ~7× the work for a notch documented as ~2×.

34cfc07a gates on userSetHits. I considered adding a second, deliberate door at high effort
and rejected it on evidence: notch +2 is measured inert on an independent 60-cell panel
(1958f211), corroborating this study's own 125-/131-/173-tip ties from a different regime.
.IwRatchetDepth() still follows the ladder-raised value, since upstream states that coupling is
the point of the multiplier.

A provenance test pins it by holding every value equal — rung 5 reaching targetHits = 20 by
ladder vs a caller naming 20 — so only the source of the number differs.

Verification

test-reach-escalation.R 16/16 blocks pass (0 failed, 0 error); test-iw-ratchet-depth.R 18/18.
Run against a temp-lib R CMD INSTALL, not load_all.

Known gaps, stated rather than hidden

  • Measured under equal weights only; applying it under implied weights and profile parsimony
    is a deliberate extrapolation (the levers are scorer-agnostic machinery) but is unmeasured.
  • The tip sample jumps 173 → 4062, so 174–4061 tips is untested.
  • The poolSuboptimal leak guard is tested, but the leak was only ever probed on a cell with
    zero replicates — the wrong cell for it. Unmeasured, not disproved.

🤖 Generated with Claude Code

ms609 and others added 17 commits July 30, 2026 16:54
Extends the existing `targetHits` escalation beyond ratchet depth.  A caller
who at least doubles `targetHits` has asked to keep searching well past
ordinary convergence, so under `thorough`/`large` also deepen the
per-replicate perturbation itself: drift 25 cycles, the auto/deep reweighting
kick, a post-ratchet sectorial pass, and (internally) near-optimal pool
retention for intraFuse.

The ratio `targetHits / defaultHits` is factored out into
`.TargetHitsEscalation()` and `.IwRatchetDepth()` now reads it too, so the two
escalations share one signal rather than each deriving its own.  Scoped to
`thorough`/`large`, matching `.IwRatchetDepth`: `sprint`/`default` document
themselves as shallow and have their own implied-weights operating point
(`.iwStopPackage`), which this must not disturb.  Unlike the ratchet
deepening, this applies under any scoring regime.

Calibrated on 25 training matrices x 5 seeds = 125 cells, both arms at the
same raised `targetHits` so only these levers differ, no cell truncated at its
wall cap.  Strict paired wins by tier: small 0/0/35 tie, medium 0/0/35 tie,
large 1 better/1 worse/33 tie, xlarge 5 better/0 worse/15 tie.  All five
xlarge wins are one matrix, project4284 at 4062 tips, improved on all five
seeds by 1-9 steps; the other xlarge matrices (125/131/173 tips) tied.  So the
benefit is datasets too large to converge in an ordinary budget, not a size
class.  Cost: median wall x3.56 and time-to-best x2.49, but
replicates-to-best x1.00 -- entirely work per replicate, not slower
convergence.  That is why these stay out of the presets and behind an explicit
signal.

`ratchetCycles` is deliberately excluded: depth belongs to `.IwRatchetDepth`,
which scales it continuously against a 36-matrix calibration, and a flat value
here would clobber it under implied weights.  Equal weights loses nothing --
a 68-matrix comparison found no equal-weights reach gain from ratchet depth.

`poolSuboptimal` is raised only as an internal aid (intraFuse needs
suboptimal recipients).  Because `collapse = FALSE` returns the pool verbatim,
the return path now drops back to the best score when the escalation raised it,
so an escalated search cannot silently return trees worse than
`attr(, "score")`.  A caller's own `poolSuboptimal` is untouched.

Caveat recorded deliberately: the A/B measured seven levers ungated via
`strategy = "auto"`; six ship, gated.  The evidence for the shipped
configuration is a tier decomposition of that run -- the xlarge win is under
`large` and preserved, and the excluded tiers measured 0 better / 0 worse, so
the gate should remove only cost -- not a direct measurement of it.  The A/B
also ran under equal weights only, so use under implied weights and profile
parsimony is a deliberate but unmeasured extrapolation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…h study

Commits the A/B harness and its findings so the evidence behind the escalation
is reproducible rather than living in scratch.  Raw per-cell CSVs stay
gitignored, as elsewhere in dev/benchmarks; the markdown is the durable record.

The findings state plainly what the study does and does not support: the
general-pool win is one 4062-tip matrix (all five of its seeds, 1-9 steps), not
a property of the >=121-tip tier, and the 20-173 tip range measured 0 better /
0 worse at ~3.5x the wall -- which is the argument for gating rather than
defaulting.  Also records that the shipped six-lever gated configuration was
never A/B'd as such (the run tested seven, ungated), and that the whole study
was equal-weights only.

The hard-tail section records the vehicle comparison for a future reach lever:
an external block loop (5/16) and the in-engine TS_POOL_RESEED (4/16) are
statistically indistinguishable, reaching the floor ~25-31% of the time either
way, and a supplied starting tree seeds only replicate 0 -- so re-solving the
incumbent is a restart-volume phenomenon that outerCycles cannot substitute for.

Includes four methodological traps that cost time here: maxSeconds is not the
search budget (enumTimeFraction reserves 10%, and identical elapsed times across
seeds is the truncation signature); whole-suite runs need test_local() because
several files call internals unqualified; a top-level skip_on_cran() reports
"0 pass 0 fail" which is not a pass; and attributing a suite failure needs a
pristine worktree at the parent commit, not a single-file swap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The paired table selected arms by the literal names "base"/"deltas", so a
variant harness with a differently-named arm produced an EMPTY comparison
rather than an error -- a silent wrong answer. Names are now taken from the
data, with "base" preferred as the baseline when present.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Confirmation A/B of the form that actually ships (six levers, gated), job
18127149: 11 large/xlarge MBANK_FIXED_SAMPLE matrices x 5 seeds. Paired final
score 8 better / 1 worse / 46 tie, no tier regression -- SHIP confirmed.

Record what the numbers do NOT say:

- The entire effect is 2 of 11 matrices (project4284 5/5; project2771 3/1/1).
  The "xlarge 20/20 vs 15/20" line is one matrix, as in the ship gate. Reach
  against a union-best-across-arms target is self-referential, so it inflates
  the loss count; paired counts are the statistic.
- project4284 won having completed ZERO replicates -- one deep replicate, not
  more restarts. That is the opposite mechanism to the hard-tail study's
  restart-volume conclusion; both recorded, neither generalised.
- project2771 is this battery's high-variance matrix (it also produced the ship
  gate's single loss); single cells there are noise.

Fix a real bug in the committed analyzer: the truncation flag tested
`wall >= 0.95 * cap_s`, but the engine stops at `maxSeconds * (1 -
enumTimeFraction)` = 0.90 * cap_s, so it reported 1 of 110 deadline-bound ab6
cells when the true count was 59 -- the very trap documented three sections
above it in the FINDINGS. The harness now records enum_time_fraction and the
analyzer derives the deadline from it.

Deadline-bound is not spoiled: when both arms stop at one deadline the cell is a
valid equal-wall comparison (the cheaper-per-rep arm gets 2.24x the replicates
and still has to win). Only asymmetry invalidates a cell. Discarding every
deadline-bound cell, as the old code did, would have dropped 40 of 55
informative cells and printed INCONCLUSIVE over a clean result. The analyzer now
classifies three ways and prints a per-matrix win/loss/tie table so a
one-matrix effect cannot read as a tier property.

Also: split FINDINGS "Not measured" into measured-on-score vs unit-tested-only
(trigger threshold, strategy scoping, userSet skip, escalatedPool filter -- all
inert in both benchmark runs, which passed levers as dots on a gate-free
engine), and note that poolReseed (v2) cannot help on the regime carrying v1's
biggest win, since it needs completed replicates to form a pool.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-ran the corrected analyzer over the original 125-cell ship-gate data. Two
statements in the FINDINGS were wrong and are now fixed:

- "No cell in either arm hit its wall cap, so nothing is a budget artefact" --
  false, and a direct artefact of the 0.95*cap_s bug. Truth: 36 base / 40 deltas
  cells stopped at the deadline, and FOUR cells are asymmetric (project2184,
  deltas at the deadline while base converged). All four tied at 563, so no
  score comparison rests on an asymmetric budget -- but the clean-sweep claim
  does not stand and should not be repeated.

- The ship-gate effect is also only 2 of 25 matrices: project4284 5/0/0 and
  project2771 1 win/1 loss/3 tie. So the `large` row's "1 better, 1 worse" is
  BOTH project2771 -- the matrix that is high-variance in both runs. That makes
  project4284 the only matrix with a clean win in either A/B, which is a
  narrower claim than the tier table alone suggests.

The x3.56 wall figure is confirmed as belonging to this run (median 3.559).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pooled across both A/Bs project2771 is 4 win / 2 loss / 4 tie over 10 seeds -- a
coin flip -- and it supplied the single loss in each run. The ab6 table listed it
beside project4284 in a way that reads as two contributing matrices, which is
inconsistent with the correction made to the ship-gate section in d269c6a.

State the surviving claim precisely instead: project4284 is 10 win / 0 loss over
10 seeds across both runs, and no matrix regresses net. That is the whole of the
positive evidence. It is enough to ship, and it is one matrix -- the largest in
the battery.

Also note in reach_escalation_ab.R's header that enumTimeFraction was added to
its argument list post-hoc (same value as the engine default, so behaviour is
unchanged) and that the 125-cell CSVs therefore predate the enum_time_fraction
column. The analyzer's "assuming the 0.10 default" note on that data is correct
behaviour, not a defect for a later session to "fix".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mal leak

Job 18128376 died after 22 minutes of search with "`tree` must be binary":
MaximizeParsimony defaults to collapse = TRUE, which contracts zero-length
branches, and TreeLength refuses a non-binary tree. Neither is at fault -- the
check was mis-specified.

Fixes, and one lesson worth more than the fix:

- Ask for collapse = FALSE, so returned trees are binary and re-scorable.
- WRITE THE TREES BEFORE VERIFYING THEM. v1 lost a whole search because it died
  in the verification step before persisting anything. The tree is the
  deliverable; the check is not.
- Score each tree in a tryCatch and report per-tree, so one bad tree cannot take
  down the run.

The corrected form also buys a measurement for free. A collapse = FALSE return is
the pool verbatim, and under a gate-free harness escalatedPool is FALSE, so the
filter that strips poolSuboptimal trees is inert -- comparing max against min of
the re-scored set therefore tests directly whether poolSuboptimal = 3 leaks
suboptimal trees to the caller. That was listed as unit-tested-only in
"Not measured". Re-run is job 18128526.

While checking that: confirmed by reading R/MaximizeParsimony.R that the
collapse = TRUE branch already restricts to scores == best_score before
collapsing, so the default path never leaked and the v1 return-path fix sits on
the only branch that needed it. Recorded, since "did the fix cover both
branches?" is the obvious question to ask of it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t size

Job 18128526 re-ran the identical deltas6 config on all 5 seeds with
collapse = FALSE, wrote the trees, and re-scored every returned tree by label.

CONFIRMED: on all five seeds the re-scored minimum exactly equals the reported
attr(res,"score"), over 100 returned trees per seed (500 trees, all scored, all
fully resolved). The ab6 headline is not an artefact of a degenerate partial
tree. Trees are now in Hamilton floors/. Best tree HELD = 354 (seed 7732); best
score ever observed = 353 (ab6 seed 7731), whose tree is lost.

QUALIFIED: the per-seed scores did NOT reproduce -- 353/356/357 came back as
355/360/363, mean 355.0 -> 357.6. Not a determinism bug: these cells complete
zero replicates and stop on the wall clock, so node speed moves the answer.
Elsewhere in the battery 46 cells tied exactly, so the instability is confined to
the matrix that never converges.

Consequences, stated in the FINDINGS:
- The 10 win / 0 loss count STANDS. Both A/Bs ran base and deltas6 sequentially
  in one R session on one node, so the pairing controls node speed; the re-run
  had no base arm and cannot add or remove wins.
- But the 2-9 step margin must NOT be quoted as an effect size. Cross-run spread
  is up to 6 steps, comparable to the effect, and the re-run's seed 7734 would
  have lost to four of five original base cells. The paired within-task design is
  load-bearing.
- The poolSuboptimal leak was not observed, and this was the wrong cell to test
  it on: all 100 trees shared one score, but with zero completed replicates the
  retention path plausibly never engaged. Still unmeasured, not disproved.

Also fix the resolution check: trees come back ROOTED, so fully resolved is 2n-2
edges, not the unrooted 2n-3. The old test reported "0/100 binary" for trees that
TreeLength -- which refuses non-binary input -- had just scored without
complaint. Observed 8122 = 2*4062-2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… 4284 record

Two findings from a verification sweep, both checked against origin/cpp-search
and the archived March library rather than inferred.

1. SHIPPING BLOCKER. 419168d removed `strategy` in favour of `effort`, a
relative offset on an internal ladder. Traced on the tip:
  .AutoRung(nTip>=120, nChar>=100) = 4; effort=+1 -> rung 5;
  hitMultiplier = 2^(5-4) = 2; line ~1124 multiplies targetHits by it
  ONLY when !userSetHits.
So on any >=120-tip dataset, effort=+1 makes targetHits/defaultHits exactly
2.0, and .reachEscalationMinRatio = 2 tests >= -- the bundle fires on a
mechanical ladder artefact, which is the precise opposite of the "user's own
signal" premise it was justified by. Rung 5 has already doubled maxReplicates,
so layering a ~3.5x-per-replicate bundle makes one notch of effort cost ~7x the
work against docs promising about twice -- and the bundle's only positive
evidence is one 4062-tip zero-replicate matrix while the 125/131/173-tip cells
all tied. Recommends gating on `userSetHits`, which the tip already computes and
whose doctrine its own roxygen states.

2. Correct the project4284 record. 354 is confirmed by five independent scorers
and is the best RECOVERABLE tree; 353 has no artefact on disk and must not be
quoted as attained. Two further caveats now recorded:

- Gap treatment must be normalised across harnesses: "-"->"?" (Fitch) vs "-" as
  a sixth level (BGS) is worth +61 on the SAME tree (354 vs 415), over 7 of 27
  characters.
- The t252 CSVs are not a comparable baseline, and NOT because of budget as I
  previously wrote. Bare AdditionTree, same data and seed, no search: 1590 under
  the archived March library vs 409 under the current one -- worse and 5x
  faster, the signature of the since-fixed union-of-finals insertion-cost bug.
  The current engine returns the same score at maxSeconds 1/5/25/45 and the
  entire 30s->1440s span is x1.068, so time was never the driver.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unsupported. Re-running the identical config on the 4062-tip matrix returned
355/354/360/363/356 against the original 353/354/356/357/355: the cross-run
spread is up to 6 steps, comparable to the quoted 1-9 step effect, because those
cells complete zero replicates and stop on the wall clock. The DIRECTION is
sound (paired within one session on one node, 10 win / 0 loss over both A/Bs),
so the sentence keeps "found shorter trees ... on all five seeds tried" and
simply stops quoting a step count.

The x3.5 wall figure is retained: it comes from the ship-gate run, which was not
deadline-bound.

NOT fixed here, deliberately: this entry and the corresponding roxygen still say
`strategy = "thorough"` / `"large"`, and upstream has REMOVED `strategy` in
favour of `effort`. That rewrite is deferred because the correct wording depends
on an unmade decision about the gate's trigger (userSetHits vs an effort door) --
rewriting now would mean rewriting again. Both files auto-merge without a
conflict marker, so this staleness is recorded in the campaign hub as part of the
blocked rebase rather than left to be discovered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…calation-gate

# Conflicts:
#	R/MaximizeParsimony.R
The escalation triggered on `targetHits / defaultHits >= 2` alone, which
upstream's `effort` ladder now satisfies by itself: .RungSpec()'s
`hitMultiplier` doubles `targetHits` at rung 5 precisely WHEN THE USER DID
NOT SET IT, so the ratio lands on exactly 2.0 and trips the gate. On any
dataset over 120 tips .AutoRung() gives rung 4, so `effort = 1` alone fired
a bundle that costs ~3.5x the per-replicate wall -- and rung 5 has already
doubled `maxReplicates`, making one notch ~7x the work against docs that
promise about 2x. .ApplyReachEscalation() now requires `userSetHits`.

That axis is also measured flat by two independent lines: this feature's own
A/B ties on every 125-, 131- and 173-tip cell, and the NA certify/effort
panel (60 cells) found notch +2 matching notch +1 on every matrix while
spending 798 replicates against 500. `.IwRatchetDepth()` still follows the
ladder-raised value -- upstream states that is the point of the multiplier --
so only this bundle changes.

Also corrects two claims in the evidence comment that my own fix to the
deadline detector falsified, and which survived in R/ after being fixed in
the FINDINGS:

- "no cell in either arm hit its wall cap" is false. The engine stops at
  `maxSeconds * (1 - enumTimeFraction)`, i.e. 0.9x: 85 cells converged in
  both arms, 36/40 stopped at that deadline, and 4 are asymmetric. Both arms
  at one deadline is still a valid equal-wall comparison and all four
  asymmetric cells tied at 563, so no conclusion moves -- but the clean-sweep
  claim does not stand.
- the "1-9 steps" margin is not an effect size. Re-running those cells moved
  scores several steps in both directions, because a deadline-truncated run
  completing zero replicates stops wherever the clock lands. The win/loss
  count replicates; the gap does not.

Records the confirmation A/B (55 cells, shipped form, 0 asymmetric: 8 better
/ 1 worse / 46 tie) and states plainly that the effect is ONE matrix --
project4284 10/0 over both runs, project2771 4/2/4 (noise), nine matrices
0/0/45 -- and that project4284 won having completed zero replicates, so what
pays is depth within a replicate, not restart volume.

Docs: purges the removed `strategy = ` argument from the NEWS entry and the
roxygen (it auto-merges with no conflict marker, so it needed catching by
hand), including one pre-existing upstream entry that named it.

Tests: new provenance test holding every VALUE equal -- rung 5 reaching
targetHits = 20 by ladder vs a caller naming 20 -- so only the source of the
number differs; the ladder run must match the rung below it and the caller's
must deepen. Four end-to-end tests migrated off `strategy = `.

Verified: 16/16 test_that blocks pass (0 failed, 0 error), plus
test-iw-ratchet-depth.R 18/18, via temp-lib R CMD INSTALL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Records the decision and, more importantly, WHY the open question closed NO
rather than by preference: notch +2 is measured inert on a second, independent
panel (60 cells, NA certify/effort), which corroborates this study's own
125-/131-/173-tip ties from a different regime.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`t253_conv_gap_mbank.csv` and `t253_gap_characterization.md` publish an n=23
Spearman analysis derived from t252 cells, and t252 ran on an engine whose
Wagner addition was several-fold worse (project4284: bare AdditionTree, same
data and seed, no search -- 1590 March vs 409 current). If that gap is broad,
every t252 start tree is systematically bad and the analysis must be RETRACTED;
if it is project4284's alone, one row needs ANNOTATING. Nothing on record
distinguishes those, so this measures bare `AdditionTree` on all 25
MBANK_FIXED_SAMPLE matrices under both engines.

Three confounds are designed out, each of which would fake a result:

- PREPROCESSING: the libraries ship different TreeTools versions, so `prep`
  builds every dataset ONCE and saves RDS; both engines read the same object.
- THE SCORER: `score` re-scores every tree under ONE library, so numbers differ
  only by tree quality -- not by scorer version.
- GAP TREATMENT: both arms map "-" -> "?", since the two conventions differ by
  +61 on one tree and would dwarf the effect being measured.

Reporting is per MATRIX (medians over 3 seeds), not per (matrix, seed): seeds
within a matrix are not independent evidence about an engine.

The ratio is deliberately OLD/NEW so that "> 1" means "March built a longer
tree", which is what every caption asserts. Alphabetical engine ordering put
`cur` in the numerator and inverted every count against its own caption --
caught in the smoke test, and worth the explicit comment it now carries.

Smoke-tested end to end locally on 3 built-in matrices, including a
same-engine control that returns ratio 1.000 on all three (which also confirms
the seeding is deterministic and the scorer is applied consistently). NOT YET
RUN on Hamilton: the VPN dropped mid-session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…aries

Both engines report `Version: 2.0.0`, so the recorded version string cannot
tell them apart. If `R_LIBS` failed to take, both arms would run the SAME
library, every ratio would be 1.000, and that reads exactly like the
"annotate, not retract" answer -- a null result obtainable from a broken arm.

The addition step now records the resolved library path, and the score step
refuses to produce a decision table unless the two arms provably came from
different paths. Verified by running it against the deliberate same-library
control, where it stops with PROVENANCE FAILURE instead of reporting ratio
1.000 on all three matrices.

Submitted as Hamilton job 18183132.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…affected

Hamilton job 18183151 answers the annotate-vs-retract question, and it answers
retract. Bare `AdditionTree`, no search, identical preprocessing, 3 seeds, every
tree re-scored by one scorer, provenance asserted:

  matrices where the March engine built a LONGER tree   25 of 25
  matrices >10% longer                                  23 of 25
  median ratio (March / current)                        1.365
  range                                        1.050 -> 3.232

So project4284 (3.23x) is the EXTREME of a universal effect, not a special case
to be excused in a footnote. A per-matrix "did it converge in 30 s" proxy built
on start trees 5-223% too long cannot support the rho values t253 quotes, so the
MorphoBank half of `t253_gap_characterization.md` is retracted and
`t253_conv_gap_mbank.csv` gets a sidecar (a header comment would break every
existing reader).

Scoped deliberately: the t265 half — 8 named datasets, TNT vs TreeSearch at
120 s — does not touch the t252 engine and stands. The CONCLUSION that ntax
predicts difficulty is corroborated elsewhere and is not called false; only this
document's evidence for it is withdrawn.

Two probe defects found by running it, both of which would have shipped a
partial answer that looked complete:

- the `__` filename separator collided with `safeKey()` output for parenthesised
  keys, silently dropping 3 of 25 matrices from the first run. The separator is
  now a character safeKey cannot emit, and the score step reports coverage and
  names any missing matrix.
- both libraries report `Version: 2.0.0`, so a failed `R_LIBS` would have run
  both arms on one engine, produced ratio 1.000 everywhere, and read exactly
  like "annotate, not retract". The score step now refuses to report unless the
  arms provably resolved to different library paths.

Result CSVs are force-added past `.gitignore`'s blanket `dev/benchmarks/*.csv`,
matching the 73 result CSVs already tracked there — this one is the retraction's
evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the loop between this study's incidental finding and the t253
retraction: the 1590-vs-409 observation that started as an explanation for one
anomalous row turns out to hold on 25 of 25 matrices (median 1.365).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant