feat: tmux-backed session persistence — terminals survive server restarts#1027
feat: tmux-backed session persistence — terminals survive server restarts#1027web3dev1337 wants to merge 5 commits into
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arts Terminals now run inside per-session tmux sessions on a dedicated per-instance socket; the pty the orchestrator owns is only a tmux client. When the app server restarts (nodemon reload, update, crash) panes keep running under the tmux server, and the next createSession() for the same deterministic id re-attaches via `new-session -A` — live agents survive. Design points (issue #1025): - Env hygiene at the tmux-server choke point: CLAUDECODE / CLAUDE_CODE_ENTRYPOINT / TMUX are scrubbed so nested-session guards never trip regardless of how the orchestrator itself was launched. - Socket options make panes act like plain terminals: status off, prefix None, tmux mouse off (pane mouse-mode requests still pass through), history-limit 20000, window-size latest, 256-color outer TERM. - Kill vs detach semantics: explicit close/terminate/workspace teardown kills the tmux session so nothing leaks detached; server shutdown only detaches — that is the survival feature. Tree-kills and the per-session process limit now target the PANE pid (a child of the tmux server), which the old client-pid group kill could never reach. - Adoption backfills the session buffer from capture-pane (escape codes preserved) so the log endpoint / scrollback preload shows pre-restart history, marked delivered to avoid double-streaming. - Fail-closed fallback: Windows and tmux-less installs keep the direct node-pty path unchanged (packaged Tauri builds unaffected); disable via ORCHESTRATOR_SESSION_PERSISTENCE=0 or config.sessions.persistence. - GET /api/sessions/persistence reports managed vs orphaned sessions; orphans stay inspectable via `tmux -L agent-workspace-<port> attach`. Tests: 16 new unit tests (backend argv/quoting/env-scrub/config, session manager pid resolution, kill semantics, orphan reporting); full suite 617/617 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…;276;0c) Under tmux, the browser terminal's auto-response to tmux's outer-terminal capability probe (DA1 ESC[?1;2c / DA2 ESC[>0;276;0c, 276 = xterm.js version) round-trips back over the socket and can arrive on the tmux client's stdin after tmux's read window closes; tmux then routes the stray bytes to the active pane, where they surface as "1;2c0;276;0c" at the shell prompt. Only Claude/TUI sessions were affected (they trigger the capability negotiation); non-tmux instances never showed it. Two-part fix: - Pin the outer terminal's capabilities via `terminal-features` (xterm-256color:RGB:clipboard) so tmux never depends on the (slow, browser-round-tripped) probe response to detect truecolor/clipboard. - Strip DA report sequences (ESC[?...c / ESC[>...c) from inbound input when a session is tmux-backed. These are terminal auto-responses a user never types; tmux answers inner-app DA queries itself, so nothing legitimate depends on the echo. Cursor-position (...R) and DSR (...n) reports are deliberately preserved — some apps read those back as input. Verified: tmux self-answers inner DA queries (repro), so filtering the echo is safe. Adds 3 unit tests for stripDeviceReports; 19 persistence tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…oint commanderService.sendToSession wrote to session.pty directly, bypassing writeToSession — so the device-report stripping (and PowerShell CRLF normalization, activity/status bookkeeping) didn't apply to that path. Route it through writeToSession so every input source shares one choke point; keep the direct write only as a fallback. Adds writeToSession device-report unit tests (tmux strips, direct does not). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Follow-up fixes after live testing surfaced a tmux-only cosmetic regression: device-attribute reports leaking as Root cause (verified, not guessed): tmux answers inner-app DA queries itself, but it also probes the outer terminal (xterm.js, whose DA2 version is Fix (three parts, all shipped on this branch):
Verified end-to-end on the live 34-session test instance: sending |
Two app-side console sources that get loud with many worktrees, both describing normal operation rather than problems: - Per-worktree button-config lookups logged "using defaults" for every worktree without a custom .orchestrator-config.json (the common case). Gated behind localStorage 'debug-worktree-config'. - The fit retry warned "proposed fit still too small (0x0)" for terminals that are simply off-screen in the scrollable grid (unmeasurable, not a real failure). The exact 0x0 case is now debug-level and schedules a delayed retry so it fits when scrolled into view; genuine small-but- nonzero fit failures still warn. (The unrelated web-client-content-script.js errors are from a browser extension, not this app.) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Summary
Terminals now run inside per-session tmux sessions on a dedicated per-instance socket (
agent-workspace-<port>); the pty the orchestrator owns is only a tmux client. When the app server restarts — nodemon reload, version update, or crash — the panes keep running under the tmux server, and the nextcreateSession()for the same deterministic id re-attaches vianew-session -A. Live agents survive.Closes the "hot reload without losing sessions" half of #1025. (Reboots still rely on transcript resume — that's the other half.)
Why / how
CLAUDECODE,CLAUDE_CODE_ENTRYPOINT, andTMUXare scrubbed so the CLI's nested-session guard never trips, regardless of how the orchestrator itself was launched (the exact trap we hit withORCHESTRATOR_PORTthis cycle).history-limit 20000,window-size latest, 256-color outer TERM.capture-pane(escape codes preserved) so the log endpoint / scrollback preload shows pre-restart history, marked delivered to avoid double-streaming.ORCHESTRATOR_SESSION_PERSISTENCE=0orconfig.sessions.persistence.enabled=false.GET /api/sessions/persistencereports managed vs orphaned sessions; orphans stay inspectable viatmux -L agent-workspace-<port> attach -t <name>.Test plan
Unit: 16 new tests (backend argv/quoting/env-scrub/config/target-matching; session-manager pane-pid resolution, kill-vs-detach, orphan reporting). Full suite 617/617 green.
End-to-end on a live instance (34 real sessions, port 3000):
Tauri / Windows note
tmux is Linux/WSL/macOS only, so the packaged Windows build keeps the in-process model (feature auto-disables). A follow-up can add a broker-daemon equivalent for native Windows if desired; tracked in #1025.
🤖 Generated with Claude Code