feat: Add scope split, followups stage, and detection standing checks#11
Open
jirispilka wants to merge 2 commits into
Open
feat: Add scope split, followups stage, and detection standing checks#11jirispilka wants to merge 2 commits into
jirispilka wants to merge 2 commits into
Conversation
…ing checks Mined review feedback across apify-mcp-server PRs shows shepherd's runs keep collecting the same human findings: untested new code arms (one shipped data bug), undisclosed behavior deltas, convention/layering drift, and silence about adjacent debt. Three additions: - Standing checks gain new-arm test coverage, undisclosed-delta disclosure, and conventions/layering; reviewer scopes in the base registry carry the per-lens detail. - The design template gains a Scope split (This PR / Prerequisite refactor / Follow-ups); a non-empty prerequisite is an explicit gate decision. - Reviewers tag adjacent pre-existing defects instead of silencing them; a new followups stage compiles them with the scope-split leftovers into a ledger the human dispositions at the create-PR gate (fix-here / issue / pr-note / drop) — issues only on human approval. PR-body numbers and references must match the final oracle run. Line cap raised 450 -> 500 (now 462). Contract tests pin the new prose. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ZmSKShwVcKG2P7MY22X8k
jirispilka
marked this pull request as draft
July 18, 2026 07:59
Five refinements from a live run (apify-mcp-server #1084 stack): human-picked light re-review for trivial gate-authorized fixes; prerequisite-refactor classification must consult the repo's refactor-separation policy; conventions doc settles style questions before they reach the human; delta-mode verify on same-base re-runs; lens-fit panel nudge at every human gate (no orchestrator auto-edits). Contract tests pin the new prose. SKILL.md 462 -> 482 lines (cap 500). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018MfTtEvQ9MYhydr8qSUopw
Pijukatel
reviewed
Jul 20, 2026
| code they describe are findings; also flag AI-slop — abnormal defensive try/catch (defensive | ||
| code at trust boundaries is fine), type-escape casts (`any` or equivalent), deep nesting that | ||
| should be early returns, and other patterns inconsistent with the surrounding file. | ||
| Three more: a conditional branch or guard the diff adds must be test-exercised on both sides, |
There was a problem hiding this comment.
Three more: is somewhat strange here. I am not sure what is meant by it
| adversarial vs live-probe vs contract/consumer). **Resolve every `"auto"` model to a concrete name** (see Model | ||
| adversarial vs live-probe vs contract/consumer). At every human gate, present the panel with a | ||
| one-line lens-fit assessment per reviewer — is the lens live for this design, or structurally | ||
| muted by it (e.g. a complexity/deletion lens on a behavior-preserving move that forbids cuts)? |
There was a problem hiding this comment.
Instruction with a question mark at the end?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What we're solving
Mined human review feedback across recent apify-mcp-server PRs (#1081, #1094, #1077, #1076, #1073, #1089, #1080): shepherd runs keep collecting the same finding classes despite passing the panel — untested new code arms (one shipped a data-corruption bug, #1094), undisclosed behavior deltas, convention/layering drift, and total silence about adjacent debt and scope boundaries. One run's own review scope even instructed reviewers to ignore carried-over defects — which the human reviewer then flagged.
How
## Scope split(This PR / Prerequisite refactor / Follow-ups); a non-empty prerequisite is an explicit human gate decision. Reviewers tag adjacent pre-existing defectspre-existinginstead of silencing them; a newfollowupsstage (dispatched with fulfillment — file contract, not orchestrator memory, so a weak orchestrator model can't drop it) compiles them into a ledger the human dispositions at the create-PR gate:fix-here | issue | pr-note | drop, issues only on human approval. A hard rule forbids scopes from suppressing reporting.Line cap 450 → 500 (now 482). Contract tests pin the prose; 97 pass.
Stacked on #6-followup branch
shepherd-skill-slop-checks(PR #7) — merge that first; this PR's base is that branch, so the diff shows only this change. Part of the same effort as #9 (refiner stage) and #10 (test-coverage lens).Field test (2026-07-18)
This branch ran a full production run end-to-end: the apify-mcp-server #1084 task-machinery refactor — triage → verify → design (6 human decisions) → design gate → 3 inner iterations → final review → fulfillment → create-PR gate → two stacked PRs shipped (apify/apify-mcp-server#1105, #1106). 24 stage dispatches, ~2.36M subagent tokens. Per-feature verdicts, each grounded in the run's
_progress.mdledger:pre-existingtag + followups ledger: worked end-to-end, and the gate is live — 4 adjacent defects surfaced across 3 review rounds without blocking convergence; the human's dispositions diverged from the stage's recommendations (3× fix-here vs pr-note/drop), reopening the loop; zero issues auto-created.Second commit (80b4c2f) folds the field-test lessons back in: human-picked light re-review for trivial gate-authorized fixes; prerequisite classification must consult the repo's refactor-separation policy; the conventions doc settles style questions before they reach the human; delta-mode verify on same-base re-runs; lens-fit panel nudge at every gate (the orchestrator never edits the panel itself). Related follow-up out of scope here: #15 (local commits on a run scratch branch; gate remote writes only).
Alternatives considered
followupsstage — rejected: compiling findings across iterations needs memory and judgment, exactly what a weak orchestrator drops; a dispatched stage re-reads the persisted review files fresh.🤖 Generated with Claude Code
https://claude.ai/code/session_017ZmSKShwVcKG2P7MY22X8k
Generated by Claude Code