Research first-class Datastar and SSE support

[lukewilliamboswell]

Purpose

Add first-class typed SSE and Datastar support to basic-webserver with
Go-comparable application ergonomics and explicit bounded ownership,
admission, backpressure, compression, cancellation, observability, and
shutdown behavior.

This remains a draft because it depends on the erased-callable ownership and
repacking work in roc-lang/roc#10530.
The experiment pins exact Roc commit 5a5f4c02; CI builds that revision until
the compiler work lands and has a normal nightly artifact.

Accepted platform design

• Applications retain { init!, respond!, shutdown! } and return
  Server.stream(Sse.Source) when they need SSE.
• Sse.unfold! creates a typed retained functional machine. Each transition
  returns Emit, Wait, End, or an application error; applications never
  receive a socket, writer, task, compressor, or host wake token.
• Datastar owns canonical patch-elements, remove-elements, and patch-signals
  event names and fields, including selectors, patch modes, namespaces,
  element-scoped view transitions, reconnect metadata, and signal options.
• The host performs one transition before committing 200 text/event-stream.
  Initial admission/timeout failures become 503; unacceptable coding becomes
  406; initial panic/application/size failures become ordinary bounded 500
  responses. A successful initial Emit, Wait, or End may commit.
• Ordinary handlers and ready SSE transitions share one fixed-worker, bounded
  FIFO Roc executor. Parked sources use no Roc worker.
• SSE stream slots, maximum event bytes, Brotli lanes/operations, response
  frames, timers, and queued execution are finite host resources. Active and
  high-water SSE/Brotli gauges are exported through OpenMetrics.
• Identity and scale-profile Brotli are negotiated by the host with canonical
  SSE headers and Vary: Accept-Encoding. Brotli uses fixed workers and lanes,
  recycled scratch, explicit per-event flush, clean normal finish, and
  cancellation without finish.
• Disconnect and graceful shutdown wake parked bodies, remove queued
  transitions, reject new executor work, and drop eventual results from an
  already-running synchronous transition while keeping it accounted until it
  returns.

The enduring contract is in design.md; the application and operational guide
is docs/sse.md.

Correctness and ownership work

• Returned erased sources are consuming-moved exactly once across precommit,
  emitted-item drain, timer parking, cancellation, and normal end.
• Executor capacity and active-handler accounting are retired before waking a
  completion consumer. This prevents a one-worker/zero-queue stream from
  rejecting its own first or immediately-following transition.
• The first transition is cancellation-safe before response commitment.
• A shared shutdown-aware source cancels parked/queued committed streams and
  abandons Brotli rather than manufacturing a clean end.
• After is capped at the documented 24-hour host timer bound, preventing
  cross-platform Instant overflow.
• SSE limits are independent from handler limits: 256 streams and 1 MiB events
  by default, configurable through Server.Config.with_sse_limits.
• Retained-state documentation distinguishes server-lifetime values and
  capabilities from expired request-scoped resources such as Server.Body.

Roc versus Go evidence

The fixed-shape comparison and raw records live in
research/datastar-e2e/results/2026-08-03-operational.md and its adjacent
JSONL. Server and Python/curl clients are pinned to separate CPUs; timing
scenarios use one warmup and three samples.

• 50 parked streams produce no material ordinary-request interference.
• With 64 transition-heavy 1,000-event streams competing with 64 ordinary
  requests, Roc ordinary p99 was 13.50 ms versus Go's 221.18 ms; Go completed
  the SSE group in 221 ms versus Roc's 531 ms. The unified bounded pool trades
  some aggregate event throughput for substantially better ordinary-request
  fairness in this single-core workload.
• Across 96 concurrent two-event streams, first-to-second gap p99 was
  102.33 ms for Roc and 110.84 ms for Go against a requested 100 ms.
• Both identity and Brotli reject stream 129 with 503, recover after one
  close, and validate Brotli recovery with Content-Encoding: br.
• A concurrent 50-stream parked cancellation burst returns stream slots and
  Brotli lanes in both implementations.
• Marginal 50-to-100 stream RSS slopes are close/noisier than the former
  cold-start ratios: identity is about 45.6 KB Roc versus 40.8 KB Go; Brotli is
  about 49.3 KB Roc versus 75.5 KB Go.

Hot-path evidence remains one process/Roc allocation per pre-rendered event,
zero additional host hot allocations, and approximately Go-comparable matched
64 KiB Brotli behavior.

Validation

• 196 Rust host unit tests
• 30 specification-harness tests
• 215 Roc platform tests
• all 27 active applications format, check, test, and build
• all 53 native x64musl runtime cases pass through real listeners
• compiled Roc SSE coverage for early Wait header commitment, precommit
  failures, canonical Datastar output, identity/Brotli negotiation, and HTTP/2
• generated Rust glue matches pinned Roc 5a5f4c02
• x64musl and arm64musl host archives build on Linux; all 27 applications
  cross-build for arm64musl
• Apple/Windows host and application builds remain assigned to their native
  release-matrix jobs

Deliberately deferred optimizations

Two ideas remain possible future measurements, not gates for this design:

1. share response-frame capacity only among currently active streams; and
2. replace the complete Roc event with a structured host-framing ABI.

Neither changes the application model, and corrected marginal-memory and
operational evidence does not currently justify taking either into this
research scope. If production evidence motivates more work, unified-pool
fairness versus aggregate transition throughput is the clearer next tuning
surface.