Skip to content

Prevent post-action flashes before video holds#9

Draft
mmkal wants to merge 7 commits into
test/simplify-video-fixturesfrom
fix/video-mode-hold-state-flash
Draft

Prevent post-action flashes before video holds#9
mmkal wants to merge 7 commits into
test/simplify-video-fixturesfrom
fix/video-mode-hold-state-flash

Conversation

@mmkal

@mmkal mmkal commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What changes

Video holds no longer leak completed action frames before replaying the pre-action state.

before: empty → filled flash → empty pointer hold → filled
after:  empty → empty pointer hold → filled

The renderer now maps video-mode timestamps onto Playwright's recorder clock:

  • Finalization paints a frame outside the render range and holds it for 1.1 seconds.
  • Playwright extends its last frame by that same Node-side elapsed time, so raw duration versus close time yields the raw/annotation offset.
  • Captions, dead air, highlights, and explicit trims move together onto the raw timeline.
  • Highlight starts are floored to the measured WebM frame duration before FFmpeg trims. No guessed 50–100 ms cutoff remains.
  • The calibration paint and Playwright's synthetic final-frame tail stay outside the rendered range.

The regression decodes every rendered frame before the hold and fails if it sees the completed action state. It also rejects the untrimmed recorder tail.

Videos

The same three fixtures attached to #7, regenerated from this branch:

Playwright captions and the account flow:

captions.webm

Dead-air compression, holds, pointer movement, and highlights:

dead-air.webm

Synthetic dialog rendering and pointer click:

dialog.webm

Relationship to #8

The current heads conflict in the constants and videoPieces() sections. The tested resolution is mechanical:

  • keep this PR's recorder calibration and calibrated highlight range;
  • keep Type short fills in video mode #8's liveAction flag, raw-action rendering, and 50 ms decoder tail.

With that resolution, a live typed fill uses raw frames from its calibrated highlight start through actionEnd; ordinary holds still use the stable pre-action screenshot. The typed-fill spec, frame-ordering regression, and caption fixture pass in a synthetic merge.

Validation

  • Frame-ordering and pointer-tail stress run: 20 passed with 2 workers
  • FFmpeg video-mode suite: 16 passed
  • Full suite: 74 passed, 3 provider-gated tests skipped
  • pnpm typecheck
  • pnpm build
  • pnpm exec publint
  • Every frame of the three PR videos inspected at the native 25 fps

Codex session: 019f948f-9c73-7d02-b1d1-788391083076

@pkg-pr-new

pkg-pr-new Bot commented Jul 24, 2026

Copy link
Copy Markdown

Open in StackBlitz

pnpm add https://pkg.pr.new/middlewright@9

commit: fb8af43

mmkal added 2 commits July 24, 2026 15:48
Playwright's recorded frame timestamps trail the action clock, so raw pre-action pieces could include completed-action frames before the captured hold. Start highlight timing before capture and replace the final 100ms of raw footage with the known pre-action screenshot.
@mmkal
mmkal marked this pull request as ready for review July 24, 2026 15:14
@mmkal
mmkal marked this pull request as draft July 24, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant