feat(review-pr): incremental reviews of new commits since last review#37
Conversation
Re-reviews only the range from the last reviewed SHA to HEAD instead of the full PR diff, tracking state via the commit_id GitHub records on completed reviews. Falls back to a full review on force-push or rebase, base merge-ins, or any error, and dedupes findings against already-posted comments so full re-reviews never duplicate threads. Adds the incremental input (default true) to the action and the reusable workflow.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
One medium-severity finding in the incremental-review CLI: a writeFileSync failure in the catch fallback path could leave pr.diff partially corrupted without restoring the backup copy. All other hypotheses were verified and dismissed.
| `full diff preserved at ${preservedPath})`, | ||
| ); | ||
| setOutputs('incremental', 'ok', sha); | ||
| } catch (err: unknown) { |
There was a problem hiding this comment.
[medium] writeFileSync failure leaves pr.diff corrupted without restoring the backup
After copying diffPath to preservedPath (line 170) and then overwriting diffPath with the incremental diff (line 171), the catch block correctly reports mode=full — but it does not restore diffPath from preservedPath.
If writeFileSync at line 171 throws (e.g. disk-full or a permission error mid-write), pr.diff may be partially overwritten with incomplete content. The downstream review steps will then read a corrupted diff even though mode=full was reported, producing either a broken review or a misleading one. The good original copy exists at preservedPath but is never used to recover.
Fix: restore diffPath in the catch block before reporting mode=full:
| } catch (err: unknown) { | |
| } catch (err: unknown) { | |
| core.warning( | |
| `incremental-review failed (${err instanceof Error ? err.message : String(err)}) — falling back to full review`, | |
| ); | |
| // Restore the original diff if the incremental write may have corrupted it. | |
| try { | |
| const preservedPath = fullDiffPath(diffPath); | |
| copyFileSync(preservedPath, diffPath); | |
| } catch { | |
| // Best-effort; if restoring also fails, the full review will error too. | |
| } | |
| setOutputs('full', 'error', sha); |
| Confidence | Score |
|---|---|
| 🟡 moderate | 67/100 |
derekmisler
left a comment
There was a problem hiding this comment.
ci is green and the fail-open design is solid. two things worth a look: in src/incremental-review/index.ts around line 170, if writeFileSync throws after copyFileSync has already written pr_full.diff, the catch block warns and falls back to full mode but doesn't restore pr.diff from the backup, so a downstream consumer could read a partially-written diff. easy fix: add a copyFileSync(preservedPath, diffPath) inside the catch before returning. separately, in findingSignature() (dedupe-findings.ts:81) the regex [^*]+ matches newlines, which can pull in extra text from multi-line bold blocks and inflate the jaccard similarity denominator enough to miss a dupe. swapping it for [^*\n]+ tightens it up.
|
|
||
| const preservedPath = fullDiffPath(diffPath); | ||
| copyFileSync(diffPath, preservedPath); | ||
| writeFileSync(diffPath, result.restricted, 'utf-8'); |
There was a problem hiding this comment.
issue: if writeFileSync throws here (disk full, permission error, etc.), pr.diff is left partially written. the catch block at line 181 falls back to full mode, but it doesn't restore pr.diff from preservedPath, so anything downstream that reads pr.diff gets a corrupted file. preservedPath is already written at this point, so the fix is one line in the catch:
} catch (err: unknown) {
try { copyFileSync(preservedPath, diffPath); } catch {}
core.warning(...)
setOutputs('full', 'error', sha);
}alternatively, write to a temp file and atomically rename over diffPath.
| Confidence | 🟡 LIKELY | 70/100 |
|---|
| * Returns null when no usable text exists. | ||
| */ | ||
| export function findingSignature(body: string): string[] | null { | ||
| const bold = body.match(/\*\*([^*]+)\*\*/); |
There was a problem hiding this comment.
suggestion (non-blocking): [^*]+ matches newlines in js, so if a comment body has a bold span that wraps lines (e.g. **[high] title\ncontinued**), the captured group pulls in all lines up to the closing **. that inflates the token set and can drop jaccard similarity below 0.5, causing a missed dedupe. the fallback branch already handles this correctly (takes only the first non-empty line). tightening the regex to match the same behavior:
| const bold = body.match(/\*\*([^*]+)\*\*/); | |
| const bold = body.match(/\*\*([^*\n]+)\*\*/); |
| Confidence | 🟡 LIKELY | 62/100 |
|---|
If writeFileSync throws mid-write after the full diff was already preserved, pr.diff could be left partially written while mode=full is reported, so downstream steps would review a corrupted diff. The catch block now restores pr.diff from the preserved copy and removes pr_full.diff, whose presence would otherwise signal incremental mode to downstream steps. The restore only runs when the backup copy completed, so a partial backup can never overwrite an intact pr.diff.
…ures In findingSignature, [^*]+ also matches newlines, so a bold span wrapping across lines pulled every line up to the closing marker into the token set. That inflates the Jaccard union enough to fall below the similarity threshold and miss a duplicate. Restrict the match to single-line bold blocks; multi-line headings now use the existing first-non-empty-line fallback.
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟢 APPROVE
The incremental review implementation is well-structured with thorough fail-open semantics throughout. All error paths correctly fall back to a full review, SHA40 regex guards prevent shell injection via git arguments, and temporary files are cleaned up with force: true in finally blocks. The deduplication logic (dedupe-findings) is sound — findingSignature always deduplicates tokens before similarity scoring, and the Jaccard formula is correct for set inputs. The findLastReviewedSha ordering logic (newest wins via >= on oldest-first API responses) is consistent with the GitHub Reviews API contract and is explicitly commented. No bugs were confirmed in the new code.
derekmisler
left a comment
There was a problem hiding this comment.
solid implementation overall; the fallback coverage and test suite are thorough. two minor suggestions around signatureSimilarity's exported contract and a rename-path asymmetry in restrictDiffToFiles, both non-blocking. ci is green.
| return tokens.length > 0 ? [...new Set(tokens)] : null; | ||
| } | ||
|
|
||
| export function signatureSimilarity(a: string[], b: string[]): number { |
There was a problem hiding this comment.
suggestion (non-blocking): signatureSimilarity can return > 1.0 if a has duplicate tokens. the loop increments intersection for each occurrence of a duplicate, but union = a.length + b.length - intersection uses the inflated raw length, so the ratio overflows. in production this is fine because findingSignature always deduplicates before calling this, but the exported function has no precondition documented or enforced. suggest deduping at entry:
| export function signatureSimilarity(a: string[], b: string[]): number { | |
| export function signatureSimilarity(aIn: string[], b: string[]): number { | |
| const a = [...new Set(aIn)]; | |
| const setB = new Set(b); | |
| let intersection = 0; | |
| for (const token of a) { | |
| if (setB.has(token)) intersection++; | |
| } | |
| const union = a.length + setB.size - intersection; | |
| return union === 0 ? 0 : intersection / union; | |
| } |
| Confidence | 🟡 LIKELY | 70/100 |
|---|
Co-authored-by: Derek Misler <derek.misler@docker.com> Signed-off-by: Sayt0 <138035894+Sayt-0@users.noreply.github.com>
Implements Phase 2D (incremental review): on PRs with multiple review cycles, re-reviews only the commits pushed since the last completed review instead of the full PR diff. Saves tokens, avoids duplicate comments, and skips unchanged code. Every unsafe or ambiguous state falls back to a full review (fail-open), so a review is never blocked or degraded.
Issue mapping
commit_idGitHub records on every posted review (GET /pulls/{n}/reviews), which is the PR head SHA at posting time. Survives across workflow runs with no extra writes and cannot be edited away like an HTML comment marker. Only completed runs count (### Assessment:or LGTM bodies); timeout and failure fallbacks leave commits unreviewed.git diff <last-reviewed-sha>..HEAD, restricted at file level to the full PR diff so inline comments never anchor outside what GitHub accepts (422). Full diff fallback when no previous review exists or the base branch has been merged in (detected by comparing merge-bases at last review vs now).git cat-file -e(SHA absent from the clone) andgit merge-base --is-ancestor(history rewritten) both force a full review.src/dedupe-findings/: new comments matching an existing bot comment by file path + line proximity (tolerance 3) + finding-heading Jaccard similarity (threshold 0.5) are dropped before posting. Human comments and bot replies never suppress a finding. Runs on every posting path (staged at/tmp/dedupe-findings.js, existing comments pre-fetched by the action).incrementalinputreview-pr/action.yml(default"true") and to the reusable workflow (type: boolean, defaulttrue).falseforces a full review.Flow
Fallback reasons (all covered by unit tests)
no-previous-reviewno-new-commitsunknown-shahistory-rewrittenbase-unresolved/base-merged-innet-zero-changesdisabled/error/no-diffDesign notes
pr_full.diff; stale-thread resolution and the suggestion validator run against it (GitHub validates anchors against the full PR diff, not the incremental range).src/incremental-review/,src/dedupe-findings/) with Vitest coverage; the action YAML only orchestrates CLI calls.pr.diffcontents and to scope the assessment to the new commits.Validation
pnpm build(tsup)tsc --noEmitbiome ci .actionlinttests/*.shshell integration