Adaptive implementer tier on fix passes; pin ponytail-review to sonnet#14
Open
jirispilka wants to merge 1 commit into
Open
Conversation
Follow-up iterations solve progressively smaller findings, but model tiering priced every iteration like the first. Two changes, asymmetric by design because loop economics are asymmetric — under zero-findings convergence a reviewer's hallucinated finding costs a full fix round, so judges must not get cheaper while implementation may: - Fix pass: on iteration >=2 with only minor/nit findings open and no design-level change, an auto-resolved implementer drops one tier (sonnet -> haiku), bumping back if a finding survives its round. Explicitly pinned models never drop. Reviewer models never change mid-loop: the PASS that ends the loop must not come from a weaker judge than the FAIL that opened it. Mirrors the architect's existing revision-pass downgrade. - Default config pins ponytail-review to sonnet instead of auto (which resolved to opus on medium/large): a one-line deletion/leanness lens doesn't earn opus, and sonnet keeps the review floor. docs/shepherd-config.md updated to match; contract + config tests pin both rules; 94 tests pass; SKILL.md at 448/450 lines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016G4uCkx2btpqfGtJRAPJKd
Pijukatel
approved these changes
Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What we're solving
Follow-up iterations solve progressively smaller findings, but model tiering priced every iteration like the first — and
autowas resolving ponytail-review, a one-line deletion/leanness lens, toopuson medium/large runs.The changes are deliberately asymmetric because the loop's economics are asymmetric: under zero-findings convergence, a reviewer's hallucinated finding costs a full fix round (implementer + oracle + full re-review), and cheaper judges raise the false-finding rate — so implementation may get cheaper across iterations while judges must not.
How
minor/nitfindings open and no design-level change, an auto-resolved implementer drops one tier (sonnet→haiku), bumping back if a finding survives its round. Explicitly pinned models never drop. Reviewer models never change mid-loop — the PASS that ends the loop must not come from a weaker judge than the FAIL that opened it (verdict continuity).ponytail-reviewtosonnet: its findings are minor/nit by construction, soopusbuys nothing, whilesonnetkeeps the review floor.docs/shepherd-config.mdupdated to match; contract + config tests pin both rules.94 tests pass; SKILL.md at 448/450 lines.
Alternatives considered
haikufor ponytail — rejected: the review floor stayssonnet; a hallucinated "shrink" nit forces a full fix round, and ponytail's clarity guard is exactly the nuance weak models drop./shepherd-evaluateoverlap data should drive per-repo roster cuts (a reviewer that keeps returning zero findings is a cut candidate) — bigger savings than any model downgrade, left to run evidence rather than encoded upfront.Notes
Stacked on #13 — merge that first; this PR's base is that branch, so the diff shows only this change.
🤖 Generated with Claude Code
https://claude.ai/code/session_016G4uCkx2btpqfGtJRAPJKd
Generated by Claude Code