Skip to content

autosetup: converge the compilation-workaround loop instead of timing out - #134

Open
shellygr wants to merge 3 commits into
masterfrom
shelly/workaround-loop-convergence
Open

autosetup: converge the compilation-workaround loop instead of timing out#134
shellygr wants to merge 3 commits into
masterfrom
shelly/workaround-loop-convergence

Conversation

@shellygr

@shellygr shellygr commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

The incident

A corpus project mixing pragma solidity 0.6.4; (37 files) and 0.8.17 (16 files) burned a full
60-minute job timeout in the compilation-workaround loop without ever converging:

compiler_version_mismatch:  X requires 0.6.4              -> compiler_map[X] = solc6.4
solc_not_found_fallback:    Detected missing solc binary  -> falls back to solc8.34
compiler_version_mismatch:  X requires 0.6.4              -> compiler_map[X] = solc6.4
... 240 rounds, one certoraRun each

solc6.4 is not installed. The fallback was version-blind — it rewrote every contract pinned to the
missing binary to whatever compiler was default, with no reference to the pragma — so the
substitution could never compile and the mismatch detector re-pinned it every pass.

The existing guard could not see this: it compares a pass against its own starting state, so it
only catches a pass that changed nothing. Two workarounds that undo each other each change the conf
relative to their own baseline, and they alternate across passes.

The fixes

1. The fallback is a per-contract plan. Each contract pinned to the missing binary is offered
only compilers its pragma admits (pragma_admits -> True / False / None, where None means
the spec could not be parsed and is treated as no evidence). When a contract has no viable
substitute the run raises UnsatisfiableSolcPinError naming the contract, its pragma and the binary
to install, rather than retrying a substitution that cannot work.

2. A change ledger, as the general backstop. Every apply records (workaround, conf delta) for
the run. A pass whose changes have all been made before means something is undoing them, so the loop
stops and names them; a pass that also lands a new change is still converging. The delta deliberately
excludes the previous value — otherwise each half of an A-undoes-B cycle looks novel forever.

The regression test drives the real shape — missing pin, substitute, re-pin — with the pragma
unreadable so the pragma guard deliberately does not apply: 3 compiles instead of 273.

Review

An adversarial review of the first two commits confirmed six defects, all fixed in 311ea7e and
covered by tests: candidate ordering preferring an unrelated PATH solc over the project default; a
UnicodeDecodeError crash on a non-UTF-8 source; =X.Y.Z / ~X.Y.Z / disjunction pragma spellings;
the terminal raise firing on a pin autosetup seeded itself; certora-fixconf exiting on a traceback
and losing the fixes it had already written; and the ledger persisting across the two loop runs
fixconf makes on one manager.

Known, not fixed here

missing_library_harness cascades into nested FooHarnessHarness... harnesses when a link error
recurs, because its _harnessed_libs guard is keyed on the consumer name and the same apply replaces
that name with the harness's. It is bounded only by max_retries. Pre-existing and independent of
these commits (reproduced with the ledger disabled); the comment no longer claims that guard bounds
it. Worth its own change.

517 passed, 9 skipped; pyright clean.

🤖 Generated with Claude Code

shellygr and others added 3 commits August 8, 2026 18:00
A contract pinned to an absent compiler was rewritten to whichever compiler was
default, regardless of what its pragma allows. For an exact pragma the rewrite
cannot compile, so the next pass re-detects the mismatch and re-pins — the two
workarounds undo each other until the retry budget or the job timeout ends it.

The fallback is now a per-contract plan: each pinned contract is offered only
compilers its pragma admits, and an unreadable or unparseable pragma still takes
the first candidate. When a contract has no viable substitute the run raises
UnsatisfiableSolcPinError naming the contract, its pragma and the binary to
install, instead of retrying a substitution that cannot work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The loop compares each pass against its own starting state, so it only notices a
pass that changed nothing. Two workarounds that undo each other each change the
conf relative to their own baseline, and alternate across passes, so the loop
runs to the retry budget — one full certoraRun per pass.

Record every (workaround, conf delta) applied in the run. A pass whose changes
have all been made before means something is undoing them, so stop there and name
them. A pass that also lands a new change is still converging and continues.

missing_library_harness is exempt via progress_outside_conf: regenerating its
harness covers one more library each time, which is real progress the conf does
not show. Its own _harnessed_libs guard bounds it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six defects found by review of the two commits below it:

- Candidate order put whatever `solc` is on PATH ahead of the project's own
  default, so a wide pragma could pick an unrelated compiler. The default now
  comes first and plain solc is the last resort.
- A non-UTF-8 source (an accented byte in a header comment) raised out of
  read_pragma_from_source_file and killed the loop; it now reads as an unknown
  pragma.
- parse_pragma_constraint understands `=X.Y.Z` and `~X.Y.Z`, and reports a
  disjunction as unknown rather than mis-parsing it.
- The terminal raise is gated on a pin the conf actually carries, so a pin
  seeded from the default compiler no longer fails the run.
- certora-fixconf reports the unsatisfiable pin and still writes back the fixes
  already applied, instead of exiting on a traceback.
- The change ledger is scoped to one run of the loop: fixconf runs it twice on
  one manager, and the second run may legitimately re-apply the first's changes.

An exempt workaround no longer vetoes repeat detection for the whole pass — it
only keeps its own changes out of the ledger. The note on missing_library_harness
now states what its guard keys on and that max_retries is what bounds it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant