Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/contracts/src/cli-flags.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ export type CliFlags = CloudProviderProfileFields &
replayShellEnv?: Record<string, string>;
replayFrom?: number;
replayPlanDigest?: string;
/** Replay: leave the session active by suppressing an authored terminal close in native .ad. */
replayKeepSession?: boolean;
failFast?: boolean;
timeoutMs?: number;
retries?: number;
Expand Down
2 changes: 2 additions & 0 deletions packages/contracts/src/client-replay.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ export type ReplayRunOptions = AgentDeviceRequestOverrides &
resumeFrom?: number;
/** The `resume.planDigest` from the divergence report `resumeFrom` came from. */
resumePlanDigest?: string;
/** Leave the session active by suppressing an authored terminal `close` in native `.ad`. */
keepSession?: boolean;
/**
* ADR 0012 decision 6, R1/R6: arms agent-supervised re-record repair
* from this replay attempt onward. Optional string value is the healed
Expand Down
1 change: 1 addition & 0 deletions packages/contracts/src/client-request.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ export type CommandExecutionOptions = Partial<ScreenshotRequestFlags> & {
replayShellEnv?: Record<string, string>;
replayFrom?: number;
replayPlanDigest?: string;
replayKeepSession?: boolean;
failFast?: boolean;
timeoutMs?: number;
retries?: number;
Expand Down
8 changes: 4 additions & 4 deletions packages/contracts/src/replay.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ export type ReplayCommandResult = {
session: string;
/**
* True iff `session` still exists in the daemon's session store when the
* response is built — i.e. the replayed script had no terminal `close`
* (ADR 0016's consumption contract). The client uses this, not script
* parsing, to decide whether an owned one-shot daemon must stay alive so
* the caller can keep addressing this session.
* response is built. This remains true when replay suppresses an authored
* terminal `close` for an explicit live-session handoff. The client uses
* this, not script parsing, to decide whether an owned one-shot daemon must
* stay alive so the caller can keep addressing this session.
*/
sessionActive: boolean;
artifactPaths: string[];
Expand Down
1 change: 1 addition & 0 deletions scripts/integration-progress-model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ function summarizeProviderScenarioFlagCoverage(files) {
['replayEnv', 'replay/test variable injection', ['env']],
['replayFrom', 'replay resume skips completed steps (ADR 0012)', ['resumeFrom']],
['replayPlanDigest', 'replay resume plan-digest preflight binding', ['resumePlanDigest']],
['replayKeepSession', 'native replay terminal-close suppression', ['keepSession']],
['failFast', 'test suite stops after first failure'],
['timeoutMs', 'wait/test timeout flags'],
['retries', 'test suite retry budget flows through request path'],
Expand Down
2 changes: 1 addition & 1 deletion src/cli/parser/cli-help.ts
Original file line number Diff line number Diff line change
Expand Up @@ -358,7 +358,7 @@ Validation and evidence:
agent-device screenshot
agent-device press 124 817
agent-device snapshot -i
Startup/CPU/memory/frame first pass: perf metrics --json (bare perf and metrics are aliases). Focused frame/jank health: perf frames --json. Memory-only sample: perf memory sample --json returns compact JSON with bounded top offenders. Heap/memgraph artifact escalation: perf memory snapshot --out heap.artifact; use --kind android-hprof on Android or --kind memgraph on supported Apple simulator/macOS app sessions. Android native profiling: perf cpu profile start|stop|report --kind simpleperf --out <path>; Android native traces: perf trace start|stop --kind perfetto --out <path>. Artifact collectors return compact state/path/size metadata only; raw heap/profile/trace files stay on disk. Treat native perf output as the agent evidence: for example, a Perfetto stop can return state=stopped, outPath=/tmp/app.perfetto-trace, sizeBytes=5392410, and method=adb-shell-perfetto while the 5.3 MB raw trace stays in the artifact. This is better than raw dumps for agents because it is stable, bounded, and keeps large artifacts out of context. heapprofd is deferred until Perfetto plumbing is available. Replay divergence and resume: a failing replay/test step returns REPLAY_DIVERGENCE with a bounded report (screen digest, ranked selector suggestions, resume). Repair app state, then resume with replay --from <n> --plan-digest <sha256> (both from the report's resume field) to continue from the failed step without re-running earlier ones; resume never re-executes skipped steps, so app state is the caller's responsibility, and it is rejected with INVALID_ARGS when the plan digest is stale, --from is out of range, or the skipped range/target touches runtime control flow. The plan digest binds the script, its includes, the effective --platform/--target, and per-action runtime/identity. Native .ad interpolation is late-bound after planning, so changing only its values keeps the digest; Maestro environment substitution occurs during compatibility parsing and can change action inputs, includes, or control expansion, so it can change the digest. --from is replay-only; test rejects it. --update/-u no longer rewrites the script (ADR 0012) — it is a no-op kept for compatibility; every divergence already carries the same ranked suggestions. Agent-supervised repair (heal-by-doing): arm replay <file>.ad --save-script[=<out>] before step 1 (armed once; --from continuations do not need it again). Every divergence carries a repairHint: record-and-heal means press the correct control via a blessed @ref from the divergence's screen.refs, recorded (no --no-record), then continue with replay --from <n+1> --plan-digest <sha256>; state-repair means the script is correct but app state is not, so fix state with --no-record actions, then replay --from <n> --plan-digest <sha256> to re-run the unchanged step; caution means something already matches the recorded selector, so a blind re-press may repeat the mistake; manual means no safe automated repair could be proven. (If close --save-script is run before a required resume, the repair is aborted and no script is written). While armed, read-only inspection YOU run to locate the repair target (snapshot -i, get attrs, find, is) is excluded from the healed script by default — no --no-record needed; the script's own authored get/is/find steps are unaffected and stay in the heal. If the step you are repairing is itself a read, pass --record on that one command so it lands in the heal (--record is accepted only on snapshot/get/is and a read-only find; it is mutually exclusive with --no-record). Ending the repair with close --save-script[=<out>] writes only the steps recorded since the arming replay as <out>, defaulting to the <file> sibling <stem>.healed.ad — review its diff before promoting it over the original.
Startup/CPU/memory/frame first pass: perf metrics --json (bare perf and metrics are aliases). Focused frame/jank health: perf frames --json. Memory-only sample: perf memory sample --json returns compact JSON with bounded top offenders. Heap/memgraph artifact escalation: perf memory snapshot --out heap.artifact; use --kind android-hprof on Android or --kind memgraph on supported Apple simulator/macOS app sessions. Android native profiling: perf cpu profile start|stop|report --kind simpleperf --out <path>; Android native traces: perf trace start|stop --kind perfetto --out <path>. Artifact collectors return compact state/path/size metadata only; raw heap/profile/trace files stay on disk. Treat native perf output as the agent evidence: for example, a Perfetto stop can return state=stopped, outPath=/tmp/app.perfetto-trace, sizeBytes=5392410, and method=adb-shell-perfetto while the 5.3 MB raw trace stays in the artifact. This is better than raw dumps for agents because it is stable, bounded, and keeps large artifacts out of context. heapprofd is deferred until Perfetto plumbing is available. Replay divergence and resume: a failing replay/test step returns REPLAY_DIVERGENCE with a bounded report (screen digest, ranked selector suggestions, resume). Repair app state, then resume with replay --from <n> --plan-digest <sha256> (both from the report's resume field) to continue from the failed step without re-running earlier ones; resume never re-executes skipped steps, so app state is the caller's responsibility, and it is rejected with INVALID_ARGS when the plan digest is stale, --from is out of range, or the skipped range/target touches runtime control flow. The plan digest binds the script, its includes, the effective --platform/--target, and per-action runtime/identity. Native .ad interpolation is late-bound after planning, so changing only its values keeps the digest; Maestro environment substitution occurs during compatibility parsing and can change action inputs, includes, or control expansion, so it can change the digest. --from is replay-only; test rejects it. Native .ad session takeover: replay <file>.ad --keep-session suppresses exactly an authored terminal close and returns the surviving session for continued commands; interior closes still run, close-less scripts are unchanged, and test/Maestro reject the option. --update/-u no longer rewrites the script (ADR 0012) — it is a no-op kept for compatibility; every divergence already carries the same ranked suggestions. Agent-supervised repair (heal-by-doing): arm replay <file>.ad --save-script[=<out>] before step 1 (armed once; --from continuations do not need it again). Every divergence carries a repairHint: record-and-heal means press the correct control via a blessed @ref from the divergence's screen.refs, recorded (no --no-record), then continue with replay --from <n+1> --plan-digest <sha256>; state-repair means the script is correct but app state is not, so fix state with --no-record actions, then replay --from <n> --plan-digest <sha256> to re-run the unchanged step; caution means something already matches the recorded selector, so a blind re-press may repeat the mistake; manual means no safe automated repair could be proven. (If close --save-script is run before a required resume, the repair is aborted and no script is written). While armed, read-only inspection YOU run to locate the repair target (snapshot -i, get attrs, find, is) is excluded from the healed script by default — no --no-record needed; the script's own authored get/is/find steps are unaffected and stay in the heal. If the step you are repairing is itself a read, pass --record on that one command so it lands in the heal (--record is accepted only on snapshot/get/is and a read-only find; it is mutually exclusive with --no-record). Ending the repair with close --save-script[=<out>] writes only the steps recorded since the arming replay as <out>, defaulting to the <file> sibling <stem>.healed.ad — review its diff before promoting it over the original.
Recording: record start/stop. The default scope is app and expects an active session created by open <app>; this keeps app proof videos tied to the intended app session. Use record start --scope device/system to explicitly request whole-screen capture where the selected backend supports it, such as recordings that intentionally span multiple apps, home screen, settings, or app transitions. Use --max-size to cap the longest edge and --quality medium|high to choose output quality across Android and Apple targets. By default, stop burns touch overlays into the video; use record start --hide-touches for the fastest raw recording. Android record start publishes a durable device manifest. Android adb screenrecord has a 180s platform limit, so longer Android recordings are returned as multiple MP4 chunks while the daemon stays alive; after daemon restart, record stop recovers only manifest-owned chunks and warns when gesture overlays are unavailable. For gesture-heavy iOS simulator proof videos, prefer --hide-touches because overlay timing depends on a stable runner session while gestures are executing. Tracing: trace start ./trace.log, trace stop ./trace.log. Paths are positional.
Stable known flow: batch ./steps.json, not workflow batch.
Inline batch JSON example:
Expand Down
8 changes: 8 additions & 0 deletions src/commands/cli-grammar/flag-definitions-workflow.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,14 @@ export const WORKFLOW_FLAG_DEFINITIONS: readonly FlagDefinition[] = [
'Replay: the plan digest a --from resume must match (from a prior divergence report); mismatch, ' +
'edits, or include/platform-expansion changes fail INVALID_ARGS before any action',
},
{
key: 'replayKeepSession',
names: ['--keep-session'],
type: 'boolean',
usageLabel: '--keep-session',
usageDescription:
'Replay: leave the session active by suppressing exactly an authored terminal close in a native .ad script; replay only, not test or Maestro YAML',
},
{
key: 'replayMaestro',
names: ['--maestro'],
Expand Down
1 change: 1 addition & 0 deletions src/commands/cli-grammar/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ export type CommandInput = Omit<InternalRequestOptions, 'batchSteps' | 'target'>
from?: PointInput;
include?: CliFlags['networkInclude'];
kind?: string;
keepSession?: boolean;
locator?: string;
mode?: 'in-app' | 'system' | 'full' | 'limited';
button?: ClickButton;
Expand Down
1 change: 1 addition & 0 deletions src/commands/command-flags.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ function buildFlags(options: InternalRequestOptions): CommandFlags {
replayShellEnv: options.replayShellEnv,
replayFrom: options.replayFrom,
replayPlanDigest: options.replayPlanDigest,
replayKeepSession: options.replayKeepSession,
failFast: options.failFast,
timeoutMs: options.timeoutMs,
retries: options.retries,
Expand Down
20 changes: 20 additions & 0 deletions src/commands/replay/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,26 @@ describe('replay resume (ADR 0012 decision 4 / migration step 5)', () => {
});
});

describe('replay --keep-session', () => {
test('projects the CLI flag through structured replay input and the daemon request', () => {
const input = replayCliReader(['./checkout.ad'], flags({ replayKeepSession: true }));
expect(input).toMatchObject({ path: './checkout.ad', keepSession: true });
expect(replayDaemonWriter(input)).toMatchObject({
command: 'replay',
positionals: ['./checkout.ad'],
options: { replayKeepSession: true },
});
expect(replayCommandMetadata.inputSchema.properties).toHaveProperty('keepSession');
});

test('test exposes and forwards no keep-session option', () => {
const input = testCliReader(['./suite.ad'], flags({ replayKeepSession: true } as never));
expect(input).not.toHaveProperty('keepSession');
expect(testCommandMetadata.inputSchema.properties).not.toHaveProperty('keepSession');
expect(testDaemonWriter(input).options).not.toHaveProperty('replayKeepSession');
});
});

describe('replay --save-script arming (ADR 0012 decision 6, R1/R6)', () => {
test('reads --save-script as a boolean flag', () => {
expect(replayCliReader(['./checkout.ad'], flags({ saveScript: true }))).toMatchObject({
Expand Down
8 changes: 7 additions & 1 deletion src/commands/replay/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ export const replayCommandMetadata = defineFieldCommandMetadata(
// neither field — it must stay a full, deterministic suite run.
resumeFrom: integerField(),
resumePlanDigest: stringField(),
keepSession: booleanField(
'Leave the session active by suppressing exactly an authored terminal close in native .ad.',
),
// ADR 0012 decision 6, R1/R6: arms agent-supervised re-record repair
// from the first replay attempt; optional string value is the healed
// script's output path.
Expand Down Expand Up @@ -93,7 +96,7 @@ export const testCommandDefinition = defineExecutableCommand(testCommandMetadata
const replayCliSchema = {
usageOverride: 'replay <path> | replay export <file.ad> [--out <path>]',
helpDescription:
'Replay a recorded session. For Maestro YAML compatibility flows, use replay <flow.yaml> --maestro and keep the target binding such as --platform ios on the replay command. A script with no terminal close leaves its session (and daemon) running until you close it or it idle-reaps — no different from a session opened interactively.',
'Replay a recorded session. For Maestro YAML compatibility flows, use replay <flow.yaml> --maestro and keep the target binding such as --platform ios on the replay command. A script with no terminal close leaves its session (and daemon) running until you close it or it idle-reaps — no different from a session opened interactively. For native .ad scripts, --keep-session suppresses exactly an authored terminal close so you can continue interactively.',
summary: replayCommandDescription,
positionalArgs: ['path'],
allowsExtraPositionals: true,
Expand All @@ -103,6 +106,7 @@ const replayCliSchema = {
...METRO_RELOAD_FLAGS,
'replayFrom',
'replayPlanDigest',
'replayKeepSession',
'timeoutMs',
'out',
'saveScript',
Expand Down Expand Up @@ -144,6 +148,7 @@ export const replayCliReader: CliReader = (positionals, flags) => ({
bundleUrl: flags.bundleUrl,
resumeFrom: flags.replayFrom,
resumePlanDigest: flags.replayPlanDigest,
keepSession: flags.replayKeepSession,
saveScript: flags.saveScript,
force: flags.force,
});
Expand Down Expand Up @@ -175,6 +180,7 @@ export const replayDaemonWriter: DaemonWriter = (input) =>
replayShellEnv: collectReplayClientShellEnv(process.env),
replayFrom: input.resumeFrom,
replayPlanDigest: input.resumePlanDigest,
replayKeepSession: input.keepSession,
saveScript: input.saveScript,
});

Expand Down
4 changes: 2 additions & 2 deletions src/daemon/handlers/__tests__/session-replay-runtime.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ test('a close-less replay reports sessionActive: true (real producer, session st
expect(response.ok).toBe(true);
if (!response.ok) return;
expect(sessionStore.get(sessionName)).toBeDefined();
expect((response.data as { sessionActive: boolean }).sessionActive).toBe(true);
expect(response.data).toMatchObject({ sessionActive: true, replayed: 2 });
});

test('a replay whose terminal close removes the session reports sessionActive: false', async () => {
Expand Down Expand Up @@ -125,7 +125,7 @@ test('Maestro YAML uses the typed engine while .ad remains generic', async () =>
const yamlResponse = await runReplayScriptFile({
req: baseReq({
positionals: [yamlPath],
flags: { replayBackend: 'maestro', platform: 'ios' },
flags: { replayBackend: 'maestro', platform: 'ios', replayKeepSession: false },
}),
sessionName,
logPath: path.join(root, 'daemon.log'),
Expand Down
Loading
Loading