fix(daemon): distrust post-gesture stability that matches the pre-gesture baseline

[thymikee]

Summary

#1542 defect 2 (documented in #1559's body, not implemented there pending sign-off): src/daemon/post-gesture-stabilization.ts marks scroll/swipe actions for stabilization, then treats two consecutive matching AX-signature polls as proof the screen settled. On iOS's AX-free synthesized gesture lane, XCTest's tree can serve a stale-but-internally-consistent read for a window after the gesture — every poll matches the previous one, "stabilization" declares victory almost immediately, and the next interaction evaluates pre-gesture node positions.

Mechanism

• Baseline capture: markPostGestureStabilization now captures the interaction-surface signature from session.snapshot — the last-known pre-gesture capture, already available at the call site (this fires post-dispatch, pre-capture) — no extra capture added to the hot path.
• Distrust decision: decidePostGestureStabilityVerdict (pure: {needsBaselineDistrust, baselineSignature, quietSignature, elapsedMs, distrustCapMs} → 'trust' | 'distrust' | 'accept-stale') fires only once a quiet poll-to-poll match is already observed. If that quiet signature still matches the pre-gesture baseline, it is not trusted — polling continues past the normal 1.5s deadline up to a bounded 3.5s cap (STABILIZATION_DEADLINE_MS + 2_000, real margin over both the 200ms poll interval and the normal deadline — see the repo's own zero-margin flake history before touching these numbers). On cap expiry with the signature still identical, the result is accepted (a genuine no-op gesture, e.g. scroll at an edge, is the honest answer) but flagged via a new post_gesture_snapshot_stale_accept diagnostic (matchedPreGestureBaseline: true) so a stale-accept is observable in ndjson.
• Comparison is subset-tolerant, not whole-array equality (interactionSurfaceMatchesBaseline in src/daemon/interaction-outcome-policy.ts): the pre-gesture baseline and the post-gesture capture are routinely fetched with different snapshot scopes (e.g. a broad wait-driven capture vs. an interactive-only selector-resolution capture), so naive whole-array equality reports "changed" from scope drift alone. My first implementation used whole-array equality and it failed on the very first live run for exactly this reason (verdict was always trust on the first quiet match, never once catching the real staleness) — root-caused and fixed before shipping, see Adjudication below.

Platform scoping

Gated to Apple only via requiresPostGestureBaselineDistrust (isApplePlatform(device.platform)). Android's persistent helper clears its accessibility-node cache before every capture — android/snapshot-helper/src/main/java/com/callstack/agentdevice/snapshothelper/AccessibilityTreeCapture.java's capture() calls clearAccessibilityCache() (UiAutomation.clearCache() on API 34+, setServiceInfo(getServiceInfo()) fallback below it) unconditionally before every traversal, for both one-shot and persistent-session capture — so an Android post-gesture read is fresh by construction and never needs this check (#1254/#1259). Android never computes a baselineSignature at all (markPostGestureStabilization skips it), so this fix costs Android nothing — latency and semantics unchanged, live-confirmed below.

Counterfactual proof (docs/agents/testing.md)

Neutralized interactionSurfaceMatchesBaseline to return false unconditionally (reproducing "no comparable baseline evidence, ever") and reran the owning test files. 8 of 31 tests went red, including the async integration test that pins the exact live regression:

 × interactionSurfaceMatchesBaseline matches identical signatures
 × interactionSurfaceMatchesBaseline matches through a broader baseline scope when the shared element is frozen
 × interactionSurfaceMatchesBaseline matches through a broader current scope when the shared element is frozen
 × interactionSurfaceMatchesBaseline tolerates tiny rect drift on the shared element
 × decidePostGestureStabilityVerdict distrusts a quiet signature matching the baseline before the cap
 × decidePostGestureStabilityVerdict accepts a baseline-matching signature once the cap expires
 × capturePostGestureStabilizedResult keeps polling past the normal deadline when the AX tree is stuck at the pre-gesture baseline (iOS)
 × capturePostGestureStabilizedResult catches a frozen target even when the baseline came from a broader-scope capture than the post-gesture reads (iOS, live regression)

Test Files  2 failed (2)
     Tests  8 failed | 23 passed (31)

Restored the fix, reran — 31/31 green.

Live validation and adjudication

Ran on iPhone 17 (D74E0B66 — iPhone 17 Pro was lease-held; same-generation substitute, precedent from prior sessions), hardware keyboard off, pnpm build && pnpm clean:daemon, explicit --state-dir.

Check	Result
gesture-lab.ad (iOS, fresh-boot)	2/2 clean — no regression
checkout-form-android.ad + gesture-lab-android.ad (Pixel_7_CI, Release APK, freezer disabled)	2/2 clean — platform scoping confirmed safe
checkout-form.ad (iOS, fresh-boot)	Still fails at step 11 — expected residual, root-caused below

The fix demonstrably catches the literal defect described in #1559: one fresh-boot run of checkout-form.ad showed the entire post-gesture signature frozen at the exact pre-gesture baseline for the full distrust window, correctly flagged rather than silently trusted:

{"level":"warn","phase":"post_gesture_snapshot_stale_accept","command":"click","data":{"action":"scroll","attempts":45,"durationMs":21542,"matchedPreGestureBaseline":true}}

(That run used a temporarily-widened 22s cap purely to test whether more time ever resolves it — it didn't, which is why the shipped cap stays at the reasoned 3.5s: a bigger bound doesn't help this specific staleness, it only costs latency on every genuinely inert gesture.)

But checkout-form.ad still fails, for a different reason this fix correctly stops masking. A second fresh-boot repro resynced within 1.5s and differed from the pre-gesture baseline (post_gesture_snapshot_stabilized, attempts:3, durationMs:1566 — correctly trust, not distrusted) — yet the click still failed "off-screen". The raw error details show why:

"rect": {"x": 126.33, "y": 136.67, "width": 74.67, "height": 37.67},
"viewport": {"x": 18, "y": 381.0, "width": 366, "height": 109.33},
"scrollDirection": "up"

rect (the "Pickup" button) is correct — it matches the screenshot and the button's own post-scroll position. viewport — the nearest scrollable ancestor's own frame, read by findNearestScrollableAncestorRect/resolveEffectiveViewportRect in src/snapshot/mobile-snapshot-semantics.ts and treated as authoritative by the off-screen guard — is corrupted: the "Checkout form" ScrollView's frame reads (18, 381.0, 366, 109.33) post-gesture, when its true, pre-gesture frame (captured moments earlier via a clean snapshot -i) was (18, 62, 366, 729). A ScrollView's own frame does not change size when its content scrolls — this is neither the correct value nor the pre-gesture baseline value, so my distrust check (which only ever compares against the pre-gesture baseline) cannot catch it by construction: it's a third, wrong value. A same-selector retry immediately after (once postGestureStabilization clears and the daemon takes the direct-XCUIElement fast path in direct-ios-selector.ts, which bypasses the bulk snapshot walk entirely) resolves the tap instantly and correctly, confirming per-element direct reads are reliable and the corruption is specific to the bulk-snapshot-derived ScrollView ancestor frame.

Evidence preserved outside the worktree at /private/tmp/ad-defect2-artifacts/:

• manual-probe/before-scroll.png, after-scroll-0ms.png — screenshots proving the gesture visibly works (refutes an inert-gesture explanation)
• manual-probe/pre-scroll-tree.json, post-scroll-tree.json — full snapshot pairs showing the ScrollView ancestor's frame corruption alongside correct child positions
• ios-diag/ — the 45-attempt/21.8s frozen-baseline capture (quoted above)
• ios-run1/, ios-run2/ — checkout-form.ad fresh-boot failures with this fix applied
• ios-gesture-lab-run1/, ios-gesture-lab-run2/ — gesture-lab.ad 2/2
• android-run2/ — Android suite 2/2

Related: #1542 — deliberately not closed here. This fix is correct and safe as far as it goes (catches the literal frozen-tree case, zero Android cost, no regression on the passing scenario), but checkout-form.ad still does not pass, so #1542 stays open on its own merits. The residual blocker — the corrupted ScrollView-ancestor-frame read findNearestScrollableAncestorRect trusts — is a distinct defect from the one this PR addresses; a separate refusal-double-check fix (re-verifying an off-screen rejection through the direct-selector path before failing, or an equivalent) is being scoped separately.

Follow-ups

• Mutation lane: neither src/daemon/post-gesture-stabilization.ts nor src/daemon/interaction-outcome-policy.ts is in scripts/mutation/modules.ts's KERNEL_MODULES, so pnpm mutation:run was not run against them per the task's own instruction not to add lane membership unilaterally. Worth considering given both now house pure agent-facing decision functions.
• ScrollView-ancestor-frame corruption (new, tracked informally here pending its own issue): needs a design decision the same way this one did.

Gates run

pnpm typecheck && pnpm lint && pnpm format:check && pnpm check:layering && npx vitest run src/daemon/__tests__/post-gesture-stabilization.test.ts src/daemon/__tests__/interaction-outcome-policy.test.ts — all clean, chained in one run before pushing. Full npx vitest run (5363 tests) also run earlier in this branch's history: green except the pre-existing src/platforms/apple/core/__tests__/apps.test.ts contention-retry file (isolated rerun 54/54 green, unrelated to this diff).

Generated by Claude Code

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.93 MB	1.93 MB	+1.0 kB
JS gzip	619.2 kB	619.1 kB	-70 B
npm tarball	738.9 kB	739.2 kB	+298 B
npm unpacked	2.59 MB	2.59 MB	+1.0 kB

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	18.6 ms	17.9 ms	-0.7 ms
CLI --help	44.4 ms	43.6 ms	-0.9 ms

Top changed chunks:

Chunk	Raw diff	Gzip diff
dist/src/viewport-dimension.js	+5.0 kB	+1.6 kB
dist/src/internal/daemon.js	+1.3 kB	+448 B
dist/src/runtime.js	-52 B	-31 B
dist/src/context.js	+4 B	-29 B
dist/src/selector-runtime.js	-48 B	-19 B

[thymikee]

P1 — Do not treat a root-only overlap as stale-baseline evidence

interactionSurfaceMatchesBaseline returns true when any shared entry has the same rect. buildInteractionSurfaceSignature includes the application/root node, whose rect is invariant under a scroll. In the scope-drift case this PR explicitly supports, a broad pre-gesture snapshot and a narrow post-gesture selector capture can share only that root (or another fixed chrome element) after a successful scroll. We then label the fresh capture as a pre-gesture match and extend every such interaction to the 3.5 s stale-read deadline.

The new scope-drift tests keep primary-action in both signatures, so they do not cover this false-distrust shape. Please make the comparison report insufficient evidence rather than a baseline match when the overlap has no discriminating surface entry (at minimum, exclude the application root), and add a regression test with only the root shared and changed/missing content.

[thymikee]

Fixed in 191c351, replacing the previous fix (295367b) rather than patching it — the review's design correction (three-valued classifier, not a boolean with an Application special-case) landed as specified.

What changed

interactionSurfaceMatchesBaseline (boolean, "any shared entry frozen") is gone. In its place, classifyBaselineSurfaceEvidence(baseline, current): InteractionSurfaceChange reuses this module's existing three-valued vocabulary:

• 'unchanged' — at least one discriminating shared entry, all frozen → distrust/accept-stale (the intended catch)
• 'changed' — a discriminating shared entry moved → trust
• 'ambiguous' — zero discriminating shared entries (empty overlap, or overlap that's only fixed chrome) → trust, insufficient evidence, first-class outcome

decidePostGestureStabilityVerdict shrank to exactly the shape from the review:

if (!needsBaselineDistrust || !baselineSignature?.length) return 'trust';
if (classifyBaselineSurfaceEvidence(baselineSignature, quietSignature) !== 'unchanged') return 'trust';
return elapsedMs < distrustCapMs ? 'distrust' : 'accept-stale';

Discriminating is computed once per signature entry at build time (buildInteractionSurfaceEntry, new discriminating: boolean field — required, so types.ts's PostGestureStabilization.baselineSignature / PendingInteractionOutcome.preSignature shapes and two hand-built test literals in snapshot-handler.test.ts got the field too), not re-derived from the composite signature key at comparison time — the key packs identifier/label/value/type/role by naive '|'-joining free-text fields, so parsing it back out for a type check would be fragile (a label containing a literal | shifts every subsequent field). An entry is non-discriminating when:

• it's the viewport root (Application/Window) — minimal local equivalent of isViewportRoot in src/snapshot/snapshot-occlusion.ts (module-private, so reimplemented rather than exported solely for this caller; same normalizeType-based substring test, comment names the source of truth)
• it's keyboard chrome — minimal local equivalent of the [Keyboard] container check in src/core/snapshot-chrome.ts's collectKeyboardChrome (that module additionally walks parentIndex ancestor/descendant chains to classify a keyboard window's whole subtree, which a flat signature entry has no ref/parentIndex to support — catching the container itself, the actual invariant-rect anchor, is the principled minimal equivalent; comment names the source)

Tests

37 tests (interaction-outcome-policy.test.ts 17, post-gesture-stabilization.test.ts 20), including the reviewer's exact shape at three layers:

• classifyBaselineSurfaceEvidence directly — signatures sharing only the Application root (Pickup→Delivery swap) → 'ambiguous'; current = root alone → 'ambiguous'; keyboard chrome excluded from overlap; root + one real frozen element still → 'unchanged' (guards against over-excluding)
• decidePostGestureStabilityVerdict — same root-only shape → 'trust' at elapsedMs: 0 (no cap tax)
• capturePostGestureStabilizedResult (full async loop, fake timers) — same shape end to end: settles in 2 capture attempts (initial + one poll), zero stale-accepts

All prior tests (whole-array scope-drift, tiny-drift tolerance, genuine movement, frozen-target regression) kept green unchanged.

Counterfactual

Reverted classifyBaselineSurfaceEvidence to the old boolean predicate (commented out the discriminating filter) and reran:

 × classifyBaselineSurfaceEvidence is ambiguous (NOT unchanged) when a real scroll leaves only the application root shared — the reviewer-caught false-distrust shape
   Expected: "ambiguous"  Received: "unchanged"
 × classifyBaselineSurfaceEvidence is ambiguous when the current capture is the application root alone
   Expected: "ambiguous"  Received: "unchanged"
 × classifyBaselineSurfaceEvidence excludes keyboard chrome from discriminating overlap
   Expected: "ambiguous"  Received: "unchanged"
 × decidePostGestureStabilityVerdict trusts immediately when a real scroll leaves only the application root shared (no cap tax)
   Expected: "trust"  Received: "distrust"
 × capturePostGestureStabilizedResult trusts immediately (no cap tax) when a real scroll leaves only the application root shared — #1563 review regression
   Error: Test timed out in 5000ms.

 Test Files  2 failed (2)
      Tests  5 failed | 32 passed (37)

The async test didn't just fail an assertion — it timed out, because the boolean predicate genuinely extended the interaction to the 3.5s distrust cap that the test's 1s fake-timer advance never covers. That's the concrete "extends to cap" failure the review predicted. Restored, 37/37 green.

Gates

pnpm typecheck && pnpm lint && pnpm format:check && npx fallow audit --base origin/main && npx vitest run src/daemon/__tests__/post-gesture-stabilization.test.ts src/daemon/__tests__/interaction-outcome-policy.test.ts — all clean, chained in one run before pushing.

Generated by Claude Code

[thymikee]

Reviewed exact head 191c351; not merge-ready yet.

1. isKeyboardChromeKind excludes only the [Keyboard] container, while the established collectKeyboardChrome logic classifies the whole keyboard window/subtree. With a keyboard visible, a successful scroll plus scope drift can leave only stable keyboard descendants (keys/assistant controls/wrappers) in the overlap; the new classifier calls that unchanged and imposes the 3.5s stale cap on a fresh result. Reuse/export structural chrome classification or retain ancestry in the signature, and add a keyboard-descendant-only overlap counterfactual that trusts immediately.

2. post-gesture-stabilization.test.ts is now about 550 LOC, beyond the repository’s 500-line extraction tripwire. Split the pure verdict/classifier coverage and shared fixtures into focused sibling test modules.

The intended frozen-baseline route and counterfactual are otherwise meaningful, and the supplied device evidence supports that narrower fix.

[thymikee]

Both findings fixed in 12a7cae.

Finding 1: keyboard descendants, not just the container

isKeyboardChromeKind checked only normalizeType(node.type) === 'keyboard' — the container itself. collectKeyboardChrome (src/core/snapshot-chrome.ts) classifies the whole keyboard window/subtree, including the "Next keyboard"/"Dictate" assistant buttons that are documented siblings of the container, not descendants — a container-descendant walk alone provably misses them.

Fixed by reusing the real source of truth rather than retaining ancestry in the signature: exported collectKeyboardChromeRefs(nodes): ReadonlySet<string> from snapshot-chrome.ts (= collectKeyboardChrome(nodes).refs, no Android union — this caller has no appBundleId in scope and only needs the iOS half). buildInteractionSurfaceSignature computes it once per signature build (needs the whole tree for the ancestor/descendant walk, same as collectKeyboardChrome itself) and threads it into buildInteractionSurfaceEntry, so:

discriminating: !isViewportRootKind(node) && !(node.ref !== undefined && keyboardChromeRefs.has(node.ref))

No new export needed beyond the one narrow predicate; picked the smaller of the two options since collectKeyboardChrome's ancestor/descendant walk only needs the flat node list, not per-entry ancestry.

Finding 2: test file split

post-gesture-stabilization.test.ts was 550 LOC (past AGENTS.md's 500-line tripwire, "Tests are not exempt"). Split by subject:

Module	Contents	LOC
post-gesture-stabilization.test.ts	markPostGestureStabilization + the async capturePostGestureStabilizedResult loop	381
post-gesture-stabilization-verdict.test.ts (new)	pure decidePostGestureStabilityVerdict coverage	208
post-gesture-stabilization-fixtures.ts (new, non-test)	shared fixtures (pickupSnapshot, deliverySnapshot, applicationRootNode, keyboardWindowNodes, makeSession)	129
interaction-outcome-policy.test.ts	unchanged home of classifyBaselineSurfaceEvidence, grew with the new regression	413

Assertions unchanged, only relocation (plus the new tests below).

Tests

The reviewer's exact regression at three layers — a shared overlap consisting only of keyboard descendants (a key + the "Next keyboard" sibling button, not the container) plus real content that changed (Pickup → Delivery, a genuine successful scroll):

• classifyBaselineSurfaceEvidence directly → 'ambiguous'
• decidePostGestureStabilityVerdict → 'trust' at elapsedMs: 0
• capturePostGestureStabilizedResult (full async loop) → settles in 2 capture attempts, zero stale-accepts, no cap tax

Counterfactual

Reverted isNonDiscriminatingSurfaceNode to the container-only check (normalizeType(node.type) === 'keyboard') and reran:

 × classifyBaselineSurfaceEvidence excludes keyboard DESCENDANTS and window SIBLINGS, not just the container, from discriminating overlap
   Expected: "ambiguous"  Received: "unchanged"
 × decidePostGestureStabilityVerdict trusts immediately when the overlap is only keyboard descendants, not the container
   Expected: "trust"  Received: "distrust"
 × capturePostGestureStabilizedResult trusts immediately (no cap tax) when the overlap is only keyboard descendants, not the container
   Error: Test timed out in 5000ms.

 Test Files  3 failed (3)
      Tests  3 failed | 37 passed (40)

Same shape as the first review's counterfactual: the async test didn't just fail an assertion, it timed out, because the container-only exclusion genuinely extended the interaction to the 3.5s distrust cap that the test's 1s fake-timer advance never covers. Restored, 40/40 green.

Gates

pnpm typecheck && pnpm lint && pnpm format:check && npx fallow audit --base origin/main && npx vitest run src/daemon/__tests__/post-gesture-stabilization.test.ts src/daemon/__tests__/post-gesture-stabilization-verdict.test.ts src/daemon/__tests__/interaction-outcome-policy.test.ts — all clean, chained in one run before pushing.

Generated by Claude Code

[thymikee]

Re-reviewed exact head 12a7cae: the prior code findings are fixed. The implementation now reuses the structural keyboard-window classifier, pins container descendants and assistant-button siblings at classifier/verdict/async-loop levels, splits the oversized test, and exact-head CI is fully green.

One readiness item remains: the new keyboard-visible scope-drift path is fixture-only; the cited iPhone run had the hardware keyboard off. Please run one iOS software-keyboard-visible scroll that changes app content and confirm it settles in two captures without post_gesture_snapshot_stale_accept/the 3.5s distrust cap. Once that exact path is evidenced, this is ready for human review.

[thymikee]

Keyboard-visible live evidence (finding requested in re-review at 12a7cae)

Live run on iPhone 17 simulator (D74E0B66-57EB-4EC1-92DC-DA0A30581FE7, same-generation substitute — iPhone 17 Pro was shut down/not in this session), ConnectHardwareKeyboard=0 so the SOFTWARE keyboard renders, pnpm build && clean:daemon, explicit --state-dir.

1. Software keyboard visible before the gesture

Tapped id="field-name" to focus the field. Screenshot (keyboard-visible.png, preserved locally — not embedded here) and snapshot both confirm the software keyboard is on screen.

snapshot -i returned a Keyboard container plus individual Key entries (Q/W/E/R/T/... rows) in the tree — not just a screenshot-level check:

total nodes: 97
Keyboard Padding-Left None
Key Padding-Left None
Key Q None
Key W None
...

2. Scroll while the keyboard stayed up, changing real app content

scroll down 0.6 — no special start-point tuning was needed; the keyboard did not auto-dismiss. Screenshot immediately after shows "Contact details" replaced by "Delivery choices" / "Preferences", keyboard still rendered at the bottom.

Node-level proof the content genuinely moved (not just visually — same elements, real rect shift, uniform ~920px scroll offset):

Label	pre-scroll rect.y	post-scroll rect.y
Contact details	233.67	-686.33 (scrolled off top)
Delivery choices	991.67 (off bottom)	71.67 (now on screen)
Preferences	1190.67 (off bottom)	270.67 (now on screen)

The post-scroll snapshot -i tree still carried 32 keyboard-related nodes, including the exact "Next keyboard" sibling button that finding 2's fix (collectKeyboardChromeRefs) exists to exclude — this run exercises that path directly, not just a fixture.

3. Pass condition — verbatim ndjson

The very next snapshot capture after the scroll (session/requests/11de0a88a23067bd.ndjson):

{"ts":"2026-08-03T12:18:47.663Z","level":"debug","phase":"post_gesture_snapshot_stabilized","session":"cwd:2b4bc58ea6d67612:default","requestId":"11de0a88a23067bd","command":"snapshot","data":{"action":"scroll","attempts":2,"durationMs":505}}

• post_gesture_snapshot_stabilized, attempts: 2 (≤ 2 — the minimum, first quiet match)
• durationMs: 505 — well under the normal 1.5s deadline, nowhere near the 3.5s distrust cap
• post_gesture_snapshot_stale_accept: zero occurrences anywhere in the session (grep -rn "post_gesture_snapshot_stale_accept" state/sessions/ | wc -l → 0)

No cap tax, no stale-accept, on the exact keyboard-visible scope-drift path the fix targets.

Artifacts

Preserved outside the worktree at /private/tmp/ad-1563-kbd-evidence/: keyboard-visible.png, post-scroll-keyboard-still-up.png, pre-scroll-keyboard-tree.json, post-scroll-snapshot-response.json, and the full session request logs under state/sessions/.

Generated by Claude Code

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-03 13:46 UTC