refactor(daemon): extract native .ad replay to packages/ad-replay (#1478 P5)

[thymikee]

refactor(daemon): extract native .ad replay to packages/ad-replay (#1478 P5)

Implements P5 per the approved design of record (proposal, binding amendment), built as staged, individually-gated commits on one branch, cutting over atomically in this PR. Step 2 of the approved sequence (behavior pinning) merged separately as #1552 and is this PR's regression net — none of those tests' assertions changed anywhere in this branch. All four findings from the first review are addressed in commits 281f33599..947b90ca3; the response comment maps them one-by-one.

The façade — final shape (structural-quality round)

packages/ad-replay exports exactly inspectAdReplay and runAdReplay as values, plus the enumerated neutral runtime vocabulary (20 types: runtime bag, outcomes/failures, manifest, evidence, and the ReplaySelectorPort family) — the maestro-shaped typed façade, pinned by the exact-symbol gate in scripts/layering/package-boundaries.test.ts (oxc static-export table; bare export * and default exports throw; plant-verified). Package exports map is exactly ".".

The façade design went through a deliberate reversal during review: an earlier round required hiding all types behind entrypoint signatures, which produced a root-side derivation shim, four daemon twin types, and a readonly→mutable copy translator; the structural-quality review reversed that rule, and this branch now exports the typed vocabulary directly — shim, twins, and translator deleted, daemon consumers use the engine's own types.

Engine internals follow the maestro precedent: runtime-port-types.ts (374 LOC, boundary vocabulary), verify-dispatch.ts (245, verify-then-dispatch policy), step-loop.ts (237, the loop + terminal-close). The daemon engine-adapter is 314 LOC (target was <300; the overage is the lastObservation per-step fix's load-bearing comment, kept deliberately). The engine's parse gate honors the port's own contract (readSelectorExpression('ordinary', [token]), with a contract-suite cell pinning adapter parity on malformed bare tokens).

• Parsing/planning/digest/resume behind the entrypoints: inspectAdReplay's manifest carries planDigest/resolveEntryIndex; resume validation stays eager by design (a rejected --from must not mutate coordinator state — pinned by test).
• Target verification runs inside the engine step loop via narrow daemon capabilities; mismatch evidence crosses as typed values (AdReplayGuardMismatchEvidence/AdReplayLandmarkMismatchEvidence), never a wire details bag.
• Neutral outcomes: no generic, no DaemonResponse in either direction; the daemon side-map preserves the exact wire object.
• Variables: the engine builds the scope and resolves each action exactly once (${VAR} interpolation is a linear scanner in ad-script after a CodeQL polynomial-redos find — 200k-trial differential fuzz, zero mismatches); scrub values are computed once per step under one name.

Shared vocabulary went to its owner

Per the review's "proper shared owner" alternative, measured per symbol: the identity vocabulary (annotationLocalIdentity, matchesLocalIdentity, matchesAncestryPrefix, LocalIdentity, identityFieldMismatches, firstAncestryMismatch) and classifyTargetBindingMatch (+ vars.ts) moved to packages/ad-script — they interpret .ad/TargetAnnotationV1 semantics and are consumed by recording-side root code (and, for vars, the Maestro path). session-replay-report-action.ts / session-replay-suggestion-ranking.ts measured as root-only consumers and moved back to the daemon (undoing an over-move). ad-script's façade additions are covered by its existing boundary row.

The selector port

Three operations (readSelectorExpression, resolveRecordedTarget — same-alternative winner+domain invariant implemented in the production adapter, lifted verbatim; buildSelectorCandidates), trafficking only in strings, kernel snapshot values, and tagged unions. Production adapter: src/daemon/replay-selector-port.ts. In-memory adapter: src/__tests__/test-utils/in-memory-replay-selector-port.ts — now honors ReplaySelectorCandidateOptions.nodes with production's exact shared-ID drop semantics. Contract suite: 9 cells × 2 adapters (18 tests), including the shared-ID demotion cell (counterfactual: ignoring nodes fails the in-memory leg while production stays green). Two AST-needing helpers (resolveReplaySuggestionCandidate, readReplaySelectorDisplayValue) are daemon-side plain exports beside the adapter — provably inexpressible through the port without leaking the AST.

Invalid-backend rejection restored

prepareReplayPlan rejects any non-maestro replayBackend with the byte-identical INVALID_ARGS message from main, before any inspection or session work. Handler-level regression test proves zero step dispatch (counterfactual: without the guard, the script executed); a companion test pins that maestro still routes.

What moved / stayed (final)

• Package: step loop + verify-and-dispatch, inspect/manifest, digest/resume internals, verification policy, selector-port type family.
• ad-script: annotation identity vocabulary, binding classification, ${VAR} vars module (shared with the Maestro path).
• Daemon (root): request admission, invalid-backend gate, P4b coordinator (sole transaction owner, ownership test untouched), capture/dispatch/publication/artifacts, wire builders consuming neutral evidence values, Maestro dispatch and format.ts routing (above both engines), report-action + suggestion-ranking, sanitizeIdentity/describeCandidate (pinned by snapshot-lines), target-identity-node.ts/target-evidence-tree.ts (shared with dispatch/recording).

Rebase: #1554's keep-session absorbed into the engine (head e6cbe6b76)

After #1554 merged, this branch rebased onto main and folded its terminal-lifecycle policy into the P5 architecture rather than keeping a parallel decider: resolveSuppressedTerminalCloseIndex/countExecutedReplayActions unified with the engine's repair terminal-close predicate inside the step loop (one OR'd suppression condition, keepSession || runtime.isRepairArmed(), checked dynamically after armStep so a first-time --save-script arm is visible); AdReplayRunRequest gained one field (keepSession); the daemon's session-replay-terminal-lifecycle.ts module is deleted (no duplicate isExecutableReplayAction anywhere); the SessionStore postcondition stays daemon-side. #1554's six unit tests pass unchanged end-to-end, plus five new package-internal runAdReplay tests cover the unified policy directly. The replayed count is now a per-dispatched-step counter (fixing the old approximation that over-counted nested-replay markers).

Decomposition: the daemon adapter is now four cohesive modules

session-replay-runtime.ts went from ~1100 lines (post-rebase) to 242 — thin orchestration only. Extracted along its natural seams: session-replay-runtime-engine-adapter.ts (473: the runtime-bag capabilities, build*Failure implementations, side-map mechanics), session-replay-runtime-plan.ts (261: backend validation, manifest inspection, resume-index resolution, Maestro routing), session-replay-runtime-session.ts (219: session preparation, repair preflight, save-script arming). Coordinator construction stays solely in the orchestrator — the ownership test passes untouched, no allowlist changes.

Two load-bearing ordering invariants discovered during the extraction are now pinned by counterfactual-verified tests: post-dispatch mismatch divergences report the pre-step artifact snapshot (engine test; counterfactual red showed the failed dispatch's own artifacts leaking in), and a rejected --from/--plan-digest never reaches prepareReplaySession's coordinator-mutating writes (plan test; counterfactual red showed pendingRecordAndHeal being cleared before the failure).

Gates (re-run at every stage; latest full chain at head e6cbe6b76)

typecheck / lint / format:check / check:layering (53 tests incl. the new exact-symbol gate) / check:replay-compat (10 mined scripts, 6 tags, 12 digest-pinned entries) / fallow — green. Full vitest: 5351/5352 at the last full run with the only failures being the documented contention-timeout class (isolate-rerun green; the two pid-liveness assertion races are fixed separately in #1556). Fallow baseline: one surgical 8-line addition for the relocated in-memory adapter; full regen deliberately rejected (would silently drop 12 unrelated stale entries).

Live evidence (exact head 947b90ca3, round 2)

Standard suites:

Leg	Scenario	Result	Time
Android (Pixel_7_CI, Release APK, android-helper 0.20.3 probe-verified)	checkout-form-android.ad	PASS	23.7s
Android	gesture-lab-android.ad	PASS	12.3s
iOS (iPhone 17 Pro sim)	gesture-lab.ad	PASS	31.0s
iOS	checkout-form.ad	BLOCKED — #1542 (known; one clean attempt; Android twin passes, engine exonerated)	15.1s

Extended evidence (Android, constructed via the CLI's own record/replay loop; verbatim log in artifacts):

• target-v1 + save-script: recorded flow with open --save-script; saved script carries # agent-device:target-v1 {"id":"refresh-metrics","role":"button",...,"verification":"verified"}. (Finding: annotations require arming at open — close --save-script alone yields selector chains without target-v1 evidence, per session-open-surface.ts arming semantics.)
• Verification green path: replay of the annotated script — 6 steps, clean pass.
• Divergence red path: replay against the wrong screen → REPLAY_DIVERGENCE, classification selector-miss (matchCount 0) — recorded target evidence did not verify, with a record-and-heal repair suggestion including --from 3 --plan-digest 8ff7b932….
• Resume: replay --from 3 --plan-digest <hash> after correcting the screen — Replayed 2 steps in 0.8s, completed cleanly.
• Save-script repair: full record-and-heal loop — armed diverging replay, live-corrected via a blessed @ref, resumed, explicit close --save-script committed the repair; repaired script ends with # agent-device:heal-complete, carries fresh "verification":"verified" annotations for every step, and replays green (6 steps, 4.0s). Behavioral observation (informational, consistent with the repair-transaction commit boundary): an armed session's --from resume does not auto-run a trailing scripted bare close; the explicit close commits.

Artifacts: /private/tmp/ad-p5-live-artifacts-r2/ (28 files incl. the verbatim command log, original/divergence/repaired scripts, per-attempt replay trees). Round-1 artifacts at the pre-review head remain in /private/tmp/ad-p5-live-artifacts/.

Residual risks

• iOS test-app Form screen: scroll-inert to synthetic gestures and recurring slow-AX deferral #1542 blocks the iOS checkout leg — the remaining ready-for-human blocker per the review.
• Selector packaging expressly NOT revisited here — remeasure at P6/W3 as a separate decision.

Later rounds (each gated green)

• Reconcile onto main absorbing feat: keep replay session active on request #1554 (keep-session folded into the engine, terminal-lifecycle module deleted) and test: make pid-liveness fixtures deterministic under load #1556/fix(daemon): reject unarmed close --save-script before teardown #1558/fix(ios): keyboard-dismiss content settle race (#1542) — partial, defect 2 needs a decision #1559.
• Review round: invalid-backend rejection restored; shared vocabulary → ad-script; neutral outcomes; verification inversion; complexity decompositions.
• Structural-quality round (9 commits @ ac5488628): typed façade + gate widening; parse-gate contract fix; step-loop three-way split; adapter 580→314; package-local resume/target-verification tests with counterfactuals; keep-session test renamed to match topology; scrub single-naming; lastObservation per-step fix; decayed comments removed; declaredScriptPlatform single-sourced in ad-script (a real engine↔daemon duplicate was confirmed and removed).

Evidence status (final): rounds 1–4 of live evidence across the branch's history proved both platform suites, the extended target-v1/divergence/resume/keep-session/save-script-rejection/${VAR} legs, and the #1558/#1563 compose checks (verbatim details in the PR comments). At the final head, a controlled 2×2 attribution matrix (main vs. this branch, same simulator) confirmed the one remaining iOS checkout flake is a main-side contention-triggered gap in the #1563/#1566 interaction — filed as #1569 with full mechanism evidence (stale_accept saturates at the cap on every run, main included; the wall-clock retry budget starves under load) — explicitly not a finding against this branch (P5 touches no iOS capture code; main passed and failed identically). Merged on maintainer decision with #1569 as the documented residual, parallel fix in progress; a final quiet-host evidence round was in flight at merge time and its results follow up on this PR.

Generated by Claude Code

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-03 15:25 UTC

[github-actions]

Size Report

Metric	Base	Current	Diff
JS raw	1.94 MB	1.94 MB	+4.9 kB
JS gzip	620.4 kB	621.3 kB	+958 B
npm tarball	740.4 kB	741.4 kB	+1.0 kB
npm unpacked	2.59 MB	2.60 MB	+4.8 kB

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	28.1 ms	29.3 ms	+1.2 ms
CLI --help	66.4 ms	67.2 ms	+0.8 ms

Top changed chunks:

Chunk	Raw diff	Gzip diff
dist/src/session.js	+6.1 kB	+1.7 kB
dist/src/agent-device-client.js	-224 B	-90 B
dist/src/cli.js	-162 B	-60 B
dist/src/cli-help.js	-94 B	-47 B
dist/src/registry.js	-27 B	-37 B

[thymikee]

Review verdict: request changes / not ready at 62cde8db9.

• [P1] Preserve invalid-backend rejection. Before this extraction, parseReplayInput rejected any .ad request whose replayBackend was set to an unknown value with INVALID_ARGS. The new native path falls through resolveReplayFormat and calls inspectAdReplay, which receives no flags; the remaining advisory parse in device selection catches and discards the error. A raw .ad replay with replayBackend: unknown`` can now execute. Restore validation in the authoritative replay path and add a real handler/router regression test proving no step dispatch occurs.

• [P1] Complete the binding façade instead of documenting deviations. The approved P5 amendment requires the package root to expose only inspectAdReplay and runAdReplay, with parsing, variables, planning, digest/resume, verification, divergence, and neutral outcomes private. packages/ad-replay/src/index.ts instead exports broad vars/digest/identity/verification/ranking/selector policy, and root handlers import those directly. That preserves the ownership smear P5 exists to remove. Move those consumers behind the two entrypoints (or keep genuinely shared recording vocabulary in its proper shared owner), then add an exact exported-symbol shape gate—not only an exports-subpath gate.

• [P1] Do not smuggle daemon wire failures through a generic. AdReplayStepRuntime<TResponse> is instantiated as AdReplayStepRuntime<DaemonResponse>, and runAdReplay returns {ok:false,response:TResponse}. Hiding the type parameter does not make the outcome neutral: opaque daemon wire/error data crosses and returns through the engine. Replace it with explicit neutral tagged execution/failure outcomes and map them to DaemonResponse only in the daemon adapter; parsing/planning/digest/resume must also occur behind runAdReplay per the accepted façade.

• [P2] Make the second selector adapter conform. The in-memory adapter ignores ReplaySelectorCandidateOptions, including nodes, while the production adapter uses it for shared-ID demotion. The dual-adapter contract only tests a unique ID, so it cannot prove the binding amendment's shared-ID-demotion cell. Make both adapters honor the same contract and add a shared-ID case that drops the ID candidate on both.

The selector port direction, package dependency direction, coordinator ownership, and current CI are otherwise clean. The PR should remain draft: #1478 still requires both exact-head live replay suites plus target-v1 verification, divergence, resume, and save-script repair evidence; the iOS checkout leg is currently blocked by #1542. No ready-for-human label until the code findings and live-readiness blockers are cleared.

[thymikee]

Re-review verdict: still request changes at 947b90ca3.

Resolved since 62cde8db9: unknown replay backends are rejected before dispatch; both selector adapters now honor shared-ID demotion; runAdReplay drives the production verification loop and coordinator ownership remains singular.

Remaining blockers:

• [P1] Enforce the approved two-entrypoint boundary. packages/ad-replay/src/index.ts still exports a broad symbol set (including formatReplaySuccessMessage and selector/runtime types), and the new exact-symbol gate pins those extras instead of the approved inspectAdReplay / runAdReplay façade. Planning/variable assembly also remains daemon-side in session-replay-runtime.ts, though P5 assigns it to the engine.
• [P1] Keep daemon wire details out of the engine. The adapter passes response.error.details verbatim as Record<string, unknown> and engine policy reads it to derive mismatch evidence. Replace this with an explicit neutral tagged mismatch payload built by the daemon adapter.
• [P1] Preserve the terminal-executable seam from feat: keep replay session active on request #1554 before integration. This head still tests terminal close against the raw final action index and returns actions.length - entryIndex; trailing non-executable replay markers can therefore dispatch a repair close, and skipped actions are counted. Rebase/sequence with feat: keep replay session active on request #1554 and retain its last-executable-action semantics and postcondition.
• [P1] Restore the exact-head quality gate. Fallow fails on new dispatchStep / prepareReplayPlan complexity and the 231-line runtime adapter; this is owner-action, not infrastructure. Split by domain question rather than baselining the new debt.

The prior live evidence is for 62cde8db9, not this head. After code blockers clear, run the required exact-head target-v1/divergence/resume/save-script suites; iOS remains explicitly blocked by #1542. Keep the PR draft and do not apply ready-for-human.

[thymikee]

All four findings addressed at head 947b90ca3 (commits 281f33599..947b90ca3), with round-2 live evidence at the exact head — point by point:

P1 invalid-backend rejection — restored at the authoritative point (prepareReplayPlan entry, before inspection or any session work), byte-identical INVALID_ARGS message vs main. Handler-level regression test proves zero step dispatch (counterfactual: with the guard removed, the script executed — expected true to be false); a companion test pins that maestro still routes.

P1 complete the façade — done, no deviations remain. Shared vocabulary went to its proper owner (packages/ad-script): the annotation identity functions, classifyTargetBindingMatch, and vars (shared with the Maestro path); report-action/ranking measured as root-only and moved back to the daemon. Everything else is package-private; parsing/planning/digest/resume run behind the entrypoints (inspectAdReplay's manifest carries planDigest/resolveEntryIndex; resume validation stays eager by design — moving it inside runAdReplay would let a rejected --from mutate coordinator state before failing). The requested exact exported-symbol gate now pins the named export list in package-boundaries.test.ts (plant-verified: a stray export fails with a clear diff).

P1 no wire smuggling — the TResponse generic is gone. The engine returns neutral tagged outcomes; mismatch evidence crosses as values into daemon-side build*Failure capabilities; the daemon adapter keeps a side-map so the client-visible DaemonResponse is the literal same object as before (replay-compat green; the only DaemonResponse mentions inside the package are comments stating its absence). Target verification itself now runs inside the engine step loop via narrow capture/classify/dispatch capabilities, with the per-step order — including the pre-step artifact-snapshot subtlety on post-dispatch mismatches — mapped before/after and preserved.

P2 second adapter conformance — the in-memory adapter honors ReplaySelectorCandidateOptions.nodes with production's exact shared-ID drop semantics (read from build.ts, not guessed); a shared-ID cell now runs on both adapters (9 cells × 2 = 18 green). Counterfactual: reverting the in-memory demotion fails its leg while production stays green.

Live-readiness — round-2 evidence at 947b90ca3 (PR body has the tables; artifacts in /private/tmp/ad-p5-live-artifacts-r2/ incl. a verbatim command log): both standard suites re-ran (Android 2/2; iOS gesture-lab PASS; checkout leg blocked by #1542, one clean attempt, Android twin passing), plus the four named behaviors constructed live via the CLI's own loop — target-v1 annotations recorded and quoted, green-path verification, a real selector-miss divergence with its record-and-heal suggestion, --from/--plan-digest resume, and the full repair loop ending in a # agent-device:heal-complete script that replays green. One informational observation from the repair leg: an armed session's --from resume does not auto-run a trailing scripted bare close (the explicit close --save-script commits) — consistent with the repair-transaction commit boundary, reported rather than worked around.

Remaining ready-for-human blocker: #1542 (iOS checkout leg). The PR stays draft until that's fixed or the corpus is repaired without weakening coverage.

Generated by Claude Code

[thymikee]

Resync complete at head e6cbe6b76, all CI green — requesting re-review.

Since the redraft: rebased onto main absorbing #1554 (its terminal-lifecycle policy folded INTO the engine step loop, unified with the repair terminal-close predicate as one suppression condition; the daemon module deleted; all six #1554 unit tests pass unchanged end-to-end, plus five new package-internal runAdReplay tests on the unified policy). Then the maintainer-approved decomposition: session-replay-runtime.ts is now 242 lines of orchestration over three extracted modules (engine adapter 473, plan 261, session prep 219); coordinator construction unmoved, ownership test untouched. Both load-bearing ordering invariants found during the extraction are pinned with counterfactual-verified tests: pre-step artifact snapshot on post-dispatch mismatches (red showed the failed dispatch's artifacts leaking), and rejected --from never reaching coordinator-mutating writes (red showed pendingRecordAndHeal cleared before failure).

Round-3 live evidence at exact head e6cbe6b76 (artifacts /private/tmp/ad-p5-live-artifacts-r3/): Android suite 2/2 (checkout 20 steps 12.7s, gesture-lab 32 steps 20.3s, helper backend/version probe-verified); iOS gesture-lab 31 steps PASS; iOS checkout blocked by still-open #1542 (one clean attempt; partial fix for it is up separately as #1559 with the remaining stabilization defect awaiting a design decision). NEW: keep-session live check — --keep-session replay ran 19/20 steps (authored terminal close suppressed), session verifiably alive afterward with populated app state, explicit close clean — proving #1554's feature rides the extracted engine. Divergence/resume spot-check: mutated script → REPLAY_DIVERGENCE selector-miss with repair hint, live correction, --from 12 --plan-digest resume completed 9 steps green.

Prior review findings all addressed in 281f33599..947b90ca3 (see the point-by-point comment above); the façade carries zero deviations with the exact-symbol gate enforcing it. Remaining ready-for-human blocker: #1542's second defect (post-gesture stabilization semantics — options documented in #1559 for a maintainer decision).

Generated by Claude Code

[thymikee]

Review findings at e6cbe6b:\n\n- [P1] Enforce the accepted two-entrypoint facade. The approved P5 amendment says packages/ad-replay exports only inspectAdReplay and runAdReplay; the current facade additionally exports formatReplaySuccessMessage, runtime/outcome types, and the full selector-port family, and the new exact-symbol gate blesses that widening. The gate also ignores export-star declarations, so it can miss future widening. Keep success formatting daemon-side, hide internal capability/selector types behind entrypoint signatures, and make the gate reject every export form.\n- [P1] Translate wire failures before the engine boundary. AdReplayDispatchOutcome carries details as a generic unknown-valued record, the daemon adapter assigns response.error.details verbatim, and engine policy parses that bag. This is still daemon wire projection crossing into the engine despite the PR's neutral-outcomes/no-generic claim. Narrow each mismatch in the adapter into explicit tagged evidence values.\n- [P1] Move variable semantics/planning behind the replay entrypoint. The daemon still assembles ReplayVarScope, interpolates actions in invokeReplayAction, and independently interpolates target verification. P5 assigns variables and planning to ad-replay; leaving these paths daemon-owned preserves duplicated orchestration/semantics.\n- [P1] Keep this draft pending exact-head live evidence. The PR body's live corpus is for 947b90c, not current head e6cbe6b, and the prescribed iOS checkout leg remains blocked by #1542. Re-run the full required corpus on the corrected exact head before readiness.\n\nThe #1554 terminal-close fold-in and current CI checks look sound, but the accepted P5 boundary/readiness gates are not yet met.

[thymikee]

Re-reviewed exact head 6a69554. No new code findings: the rebase onto #1563/#1566 is sound, and the P5 extraction's facade/adapter/engine behavior, target verification, resume ordering, artifact ordering, keep-session handling, and selector-port contracts remain intact.

Readiness is still pending exact-head practical evidence: now that #1542 is fixed, run and attach the prescribed iOS checkout corpus plus the extended target-v1/divergence/resume/save-script evidence at this head. Then refresh the PR body to remove the stale #1542 blocker and future-tense evidence language. The PR remains draft and should not receive ready-for-human yet.

[thymikee]

Post-merge evidence round (branch head 6a695544c = merged content; verbatim command logs at /private/tmp/ad-p5-evidence-FINAL/retry2/command-log.txt, 1573 lines):

Fully green: Android suite (checkout 20/20, gesture-lab 32/32, helper backend/version probe-verified), iOS gesture-lab 3/3, and all seven extended legs with device-state proofs — target-v1 annotation quoted; recorded replay green; wrong-screen REPLAY_DIVERGENCE with repair hint; --from/--plan-digest resume completing through the terminal close; --keep-session suppression with a live device status claim check; unarmed close --save-script rejection with the recovery hint; ${VAR} interpolation proven on-device (get text returned the interpolated value from the live field).

iOS checkout-form: 0/3 this round, all failures attributed to known/new main-side issues, none to this PR's content (the pre-merge 2×2 matrix proved main fails/passes identically): twice the #1569 signature (stale_accept at attempts:6-7 / cap, then the step-16 off-screen refusal — fresh ndjson quotes posted on #1569; fix in progress), once a newly-identified cross-platform app-mount race on fast/cold opens, filed as #1571 (also reproduced independently on Android in the same round, retry-green). Host contention from a concurrent session was present throughout the iOS legs and is the known #1569 trigger.

Net: the extraction's own behavior is fully evidenced; the checkout corpus's reliability now tracks #1569/#1571, both filed with mechanisms and owned separately.

Generated by Claude Code