From 6062408999dbd53420b2fc2336c7281539a6cb95 Mon Sep 17 00:00:00 2001 From: David Chen Date: Thu, 28 May 2026 10:48:27 -0700 Subject: [PATCH 001/175] chore: update skills --- .claude/skills/conformance/SKILL.md | 62 +++++ .claude/skills/dashboard/SKILL.md | 33 +++ .claude/skills/decision-guard/SKILL.md | 280 ++++++--------------- .claude/skills/parity/SKILL.md | 153 ------------ .claude/skills/porting-to-rs/SKILL.md | 326 ------------------------- .claude/skills/rust-review/SKILL.md | 236 ------------------ .claude/skills/spec-amend/SKILL.md | 34 +++ CLAUDE.md | 280 ++++++--------------- 8 files changed, 272 insertions(+), 1132 deletions(-) create mode 100644 .claude/skills/conformance/SKILL.md create mode 100644 .claude/skills/dashboard/SKILL.md delete mode 100644 .claude/skills/parity/SKILL.md delete mode 100644 .claude/skills/porting-to-rs/SKILL.md delete mode 100644 .claude/skills/rust-review/SKILL.md create mode 100644 .claude/skills/spec-amend/SKILL.md diff --git a/.claude/skills/conformance/SKILL.md b/.claude/skills/conformance/SKILL.md new file mode 100644 index 00000000..133c7d1a --- /dev/null +++ b/.claude/skills/conformance/SKILL.md @@ -0,0 +1,62 @@ +--- +name: conformance +description: "Behavioral conformance check across GraphReFly language runtimes (ts/rust/py) for the clean-slate redesign. Replaces the old structural 'parity' diff. Parity = does each runtime satisfy the wave-protocol behavior (conformance scenarios) + dispatcher contract — NOT 'do the symbol sets match'. Use after implementing/changing substrate behavior in any runtime, or when adding a new protocol rule. Authors/runs language-agnostic scenarios and updates conformance.jsonl runtime status. Triggers: 'conformance', 'cross-lang check', 'does rust match', 'parity', 'run the conformance suite', 'is the substrate behavior consistent'." +disable-model-invocation: true +argument-hint: "[rule-id | scenario-id | 'full'] [optional: runtime ts|rust|py]" +--- + +You are executing **conformance** for the clean-slate GraphReFly redesign. + +**Parity is behavioral, not structural (D24).** There is NO `Impl` symbol-set to diff and NO +cross-track-ledger. Operators / sugar / inspection are **per-language and never in parity** +(D6/D27 — graph-layer wraps everything to `(ctx)=>void` before register). The ONLY parity +surface is: **wave-protocol behavior + dispatcher contract + handle format**. Conformance = +each runtime passes the same language-agnostic scenarios. + +## Authority + +| Source | Role | +|---|---| +| `~/src/graphrefly/spec/conformance.jsonl` | The scenario registry: `{id, name, covers:[rule-id], runtimes:{ts,rust,py}, status, note}`. | +| `~/src/graphrefly/spec/rules.jsonl` | The rules scenarios pin (`covers` must resolve here). | +| `~/src/graphrefly/spec/protocol.proto` | Protocol-contract IDL (DR-2) — the light structural anchor codegen'd into each runtime's interface stub. | +| `~/src/graphrefly/formal/*.tla` | TLA+ model (γ); property tests mirror its invariants. | + +## Scope from $ARGUMENTS + +- **rule-id** (e.g. `R-diamond`) → all scenarios whose `covers` includes it. +- **scenario-id** (e.g. `C-1`) → that scenario. +- **full** → every `status:"required"` scenario. +- optional **runtime** → restrict to one arm. + +## Phase 1 — scenario integrity + +1. Load `conformance.jsonl` + `rules.jsonl`. Verify every `covers` rule-id resolves (else HALT — fix the scenario or add the rule via `/spec-amend`). +2. List the **DR-5 required hard scenarios** and their status: `C-1` cross-graph diamond, `C-2` async-result-at-paused-node, `C-3` INVALIDATE×ctx.state×onInvalidate, `C-4` mixed sync/async diamond, `C-5` PAUSE-lockset multi-source. These are the load-bearing ones — behavioral parity is a blank cheque until they're green on each shipped runtime. + +## Phase 2 — run / verify per runtime + +For each in-scope `(scenario, runtime)`: +1. Locate/author the scenario harness in that runtime's conformance test dir (language-agnostic spec → per-runtime adapter; the scenario describes observable wave behavior, not a symbol call). +2. Run it. Record outcome. +3. Update `conformance.jsonl` `runtimes.`: `"todo" | "poc-pass" | "pass" | "fail"`. +4. Mirror the invariant as a property test (fast-check ts ↔ proptest rust ↔ hypothesis py) where the rule is property-shaped (L5-Q2 / D14). + +## Phase 3 — report + +| scenario | covers | ts | rust | py | verdict | +|---|---|---|---|---|---| + +- **Behavior drift** = same scenario, different observable outcome across runtimes → this is the ONLY kind of parity gap. File it as a substrate bug in the lagging runtime (route fix via `/dev-dispatch` on that package). +- **Missing scenario** for a rule (rule's `covers_by` empty) → author it (this is the real risk under behavioral parity: untested behavior can drift silently — D24 residual). Flag via `/dashboard` Gaps (uncoveredRules). +- **NOT a gap:** a runtime having a different operator set / different sugar / different inspection ergonomics. Those are per-language by design — do not report them. + +## Phase 4 — gate + +Run `node ~/src/graphrefly/dashboard/build.mjs --check` (scenario↔rule links intact). For a runtime +to be declared "conformant", all `status:"required"` scenarios must be `"pass"` on its arm. + +## Boundaries + +Does NOT diff symbol sets (that's the retired structural model). Does NOT touch operators/sugar/inspection +(per-language). New protocol behavior must go through `/spec-amend` FIRST (scenario authored before code). diff --git a/.claude/skills/dashboard/SKILL.md b/.claude/skills/dashboard/SKILL.md new file mode 100644 index 00000000..a2903ef4 --- /dev/null +++ b/.claude/skills/dashboard/SKILL.md @@ -0,0 +1,33 @@ +--- +name: dashboard +description: "Build / check the GraphReFly internal docs dashboard (jsonl single-source -> generated HTML with progress, structure map, gaps, search). Use when the user wants to see global project state, regenerate the dashboard, run the docs consistency gate (broken links / orphans / coverage gaps), or after editing any jsonl in ~/src/graphrefly (decisions/plan/spec/sessions/guide). Triggers: 'build the dashboard', 'check docs consistency', 'what are the gaps', 'show progress', 'regenerate dashboard', 'doc gate'." +argument-hint: "[--check (gate only) | (default: build + report)]" +--- + +You are executing **dashboard** for the clean-slate GraphReFly redesign. + +**Repo:** `~/src/graphrefly` (clean-slate branch). All structured docs are jsonl (single source of truth, decision 2); the dashboard renders them into one searchable HTML view for the maintainer (decision 3). Schema contract: `~/src/graphrefly/dashboard/README.md`. + +## What this skill does + +1. **Run the generator:** + - `node ~/src/graphrefly/dashboard/build.mjs` → writes `dashboard/dashboard.html` + prints counts / gaps / broken-links / orphans report. + - `node ~/src/graphrefly/dashboard/build.mjs --check` → consistency gate only; **non-zero exit on broken links** (use as a pre-commit / CI gate). +2. **Interpret the report** for the user: + - **counts** — per-jsonl row counts (decisions/phases/rules/conformance/...). + - **gaps** — designPhases (status=design|gap) · openDecisions · deferredBacklog · uncoveredRules (no conformance) · todoConformance (runtimes=todo). This answers "哪里还有缺口". + - **broken links** — must be zero (session.locks → decision; phase.sessions → session; conformance.covers → rule; flowchart.explains → rule|D#). Legacy 3-digit D### / R# refs are external (old main), reported separately as OK. + - **orphans** — decisions referenced by no session (informational). +3. **If broken links exist:** locate the offending jsonl row, fix the id reference (or add the missing record), re-run `--check`. + +## When the jsonl changed + +After any edit to `decisions/`, `plan/`, `spec/`, `sessions/`, `guide/` jsonl — run `--check` to catch dangling references immediately (fixes P4 stale-premise + P6 link-rot). The generator is the enforcement mechanism for "single canonical, no broken cross-refs." + +## UI styling + +`build.mjs` emits a **placeholder shell** with the data model embedded. Visual design / interactive search is a separate `/frontend-design` pass — do NOT hand-style the HTML here; keep `build.mjs` focused on the data model + consistency checks. The dogfood endgame (phase CSP-8) rebuilds this dashboard *with GraphReFly itself* (jsonl producer → reactive views → HTML effect). + +## Output + +The counts + gaps + link-health report, a plain-language "where are the gaps / what's the progress" summary, and (unless `--check`) confirmation that `dashboard.html` was written. Flag any broken link as a blocker to fix before commit. diff --git a/.claude/skills/decision-guard/SKILL.md b/.claude/skills/decision-guard/SKILL.md index a228a176..78f64884 100644 --- a/.claude/skills/decision-guard/SKILL.md +++ b/.claude/skills/decision-guard/SKILL.md @@ -1,229 +1,87 @@ --- name: decision-guard -description: "GraphReFly Rust-port decision-consistency check. Loads the user's locked values/principles/invariants + the canonical D-numbered decision log + recurring decision-process patterns. Use BEFORE answering any question of the form 'is this consistent with our decisions?', 'should I pick option A/B/C?', 'what about this proposed fix?', 'is X part of our scope?', 'is this a regression on a prior decision?'. Triggers: 'decision check', 'drift check', 'align check', 'is this consistent', 'should I pick', 'what about this', 'is this in scope', 'consistency review'." -argument-hint: "[short context of what you're being asked about — paste the chat/proposal if relevant]" +description: "GraphReFly clean-slate decision-consistency check. Loads the user's locked values/principles + the unified D-numbered decision log (decisions.jsonl) + recurring decision-process patterns. Use BEFORE answering any question of the form 'is this consistent with our decisions?', 'should I pick option A/B/C?', 'what about this proposed fix?', 'is X part of our scope?', 'is this a regression on a prior decision?'. Triggers: 'decision check', 'drift check', 'align check', 'is this consistent', 'should I pick', 'what about this', 'is this in scope', 'consistency review'." +argument-hint: "[short context of what you're being asked about — paste the proposal if relevant]" --- -# decision-guard — Recall and apply locked decisions, values, invariants +# decision-guard — recall and apply locked decisions, values, invariants (clean-slate) -**Purpose.** Future conversations about the GraphReFly Rust port lose context-window state quickly. This skill is the canonical recall surface: invoke it BEFORE answering decision questions to anchor against the user's locked positions and prevent silent drift. Especially valuable when: +**Purpose.** Conversations lose context-window state quickly. This is the canonical recall +surface for the **clean-slate** redesign: invoke BEFORE answering decision questions to anchor +against the user's locked positions and prevent silent drift — especially when a chat proposes +a scope expansion mid-implementation, presents A/B/C as a fork, uses "completeness" to justify +expanding a locked slice, or builds on a premise that may be stale. -- A chat (or a subagent's chat) proposes a scope expansion mid-implementation. -- Multiple options (A/B/C, α/β/γ) are presented as a fork. -- A "completeness" argument is used to justify expanding a locked slice. -- A finding is being triaged (patch / defer / reject). -- A premise sounds plausible but might be stale (the substrate moved under it). +> **Clean-slate retired the old port model.** Do NOT reach for `rust-port-decisions.md`, +> `cross-track-ledger.md`, the `Impl` parity contract, `BindingBoundary`, the actor model, or +> 3-digit D### port decisions — those are old-`main` history. The clean-slate decision +> authority is below. -The user has repeatedly invoked patterns from this skill across sessions; relay-ready answers should cite them by name. +## Sources (load in order) -## Authority pointers (load these only if the question requires them) - -| File | What it is | +| Source | Role | |---|---| -| `~/src/graphrefly-ts/docs/rust-port-decisions.md` | **D-numbered decision log.** Each entry has Date / Context / Options / Decision / Rationale / Affects. The canonical record. | -| `~/src/graphrefly-rs/docs/migration-status.md` | Milestone + slice closing blocks; the live tracker. | -| `~/src/graphrefly-rs/docs/porting-deferred.md` | Deferred concerns registry. Findings matching an entry here → **reject silently** in /qa. | -| `~/src/graphrefly-ts/docs/cross-track-ledger.md` | TS↔Rust coupling events. Substrate-contract widening lands here BEFORE the change. | -| `~/src/graphrefly-ts/docs/implementation-plan-13.6-canonical-spec.md` | **Behavior authority** for the Rust port. Canonical spec wins over current TS impl per §11 Implementation Deltas. | -| `~/src/graphrefly-ts/archive/optimizations/cross-language-notes.jsonl` | Verified, sanctioned divergences (`divergence-*` ids). Findings matching these → reject silently. | - -## User values & principles (immovable; cite these by name) - -1. **No backward compat (pre-1.0).** Free to refactor/rename any API. No legacy shims. Memory: `feedback_no_backward_compat`. When user says "ignore legacy/backward compat" → take the structurally cleaner option without hesitation. -2. **No imperative triggers in public API.** Coordination via reactive `NodeInput` signals and message flow. Imperative methods only on L2.35 controller-with-audit primitives. Actively remove imperative paths when no caller depends. Memory: `feedback_no_imperative`. -3. **Single source of truth.** No mirroring logic across FFI boundary; no duplicate state. Core is authority on its invariants; binding/JS-side preflight = drift bait. Memory: `feedback_single_source_of_truth`. -4. **No autonomous decisions.** Surface spec↔code conflicts; don't silently pick. File-by-file review cadence for multi-file rewrites. Memory: `feedback_no_autonomous_decisions`. **Hard rule.** -5. **No implement without approval.** Decisions locked ≠ implementation approved. Wait for explicit "implement" instruction. Memory: `feedback_no_implement_without_approval`. -6. **Pre-design full decision-set before slicing.** Avoid CoreFull-style accretion (D232→D243→D244→D245 layered widening). Design facade traits' full surface ONCE. Cite spec R-IDs in test expectations. Audit user-visible semantics at design time, not in QA. Memory: `feedback_pre_design_full_decision_set`. -7. **Verify premise before greenfield.** Design-session task tables lag the code; grep named symbols + check landed markers before any 9Q. Surface stale premises as a HALT. Memory: `feedback_verify_premise_before_greenfield`. This pattern has produced multiple wins (D256 invoke_fn_with_core premise-check; D260 timing-divergence reframe). -8. **Sync internal, async at boundary — BOTH directions.** Async/Promise only at the four sanctioned edges (napi `#[napi] async fn` surface, wrapper.js public surface, `timer.rs` tokio task, `graphrefly-storage` tokio integration). Inside `graphrefly-core` / `graphrefly-graph` / `graphrefly-operators` / `graphrefly-structures`: pure sync + reactive. **The boundary applies in BOTH directions** — sync escape hatches on binding read methods callable from inside TSFN callbacks violate the invariant (D070/D077 deadlock recurrence). Read methods on Bench* napi classes must be `async fn`. Memory: extend `feedback_async_sources_binding_layer`. -9. **Consumer-pressure (D196).** No speculative substrate surface. A Rust-core symbol gets a napi binding when (a) a non-pattern consumer materializes, OR (b) a parity scenario exercises it cross-impl. "Parity scenarios are the consumer pressure signal." Applies recursively — don't add features expecting future use; wait for the test/scenario that needs them. -10. **Spec is authority.** The canonical-spec doc wins over current TS implementation. Widening must be explicit (cross-track-ledger event). Test expectations cite R-IDs (R1.3.5.a, R2.5.3, R2.6.4, etc.) — not intuition. -11. **Completeness AND discipline.** When implementation surfaces a real semantic gap, **formalize the scope expansion as a new D-number** with proper design + test plan, don't continue under the original D's banner. (Path X-via-D264 pattern, not Path X-direct or Path Y trim.) Discipline without completeness ships half-baked surfaces; completeness without discipline auto-expands scope. -12. **Long-command observation discipline.** Use `mise run gate` / `mise run run-logged` with sentinel grep (`<<>>`). Never pipe through `tail` (buffers until EOF). Never poll via `sleep` loops. Memory: `feedback_long_command_observation`. -13. **Subagent hygiene.** Synchronous verification OR teardown bg processes (kill by process group) before returning. A leaked bg process surfaces as a stale "running" entry indistinguishable from a real hang. Memory: `feedback_subagent_bg_hygiene`. -14. **Distinguish vestigial-surface from speculative-surface.** D196 ("no speculative substrate") governs **new** surface added without consumer pressure. It does **NOT** govern cleanup of surface that an earlier-locked decision RELAXED into vestigial overhead (e.g., D248 relaxed Sink Send+Sync → operator-internal `Arc>` capturing single-owner state became dead weight). Vestigial-surface removal is **decision-consistency restoration**, a separate justification class with locked precedent: D253 (SchedulingGroupId delete), D254-AUDIT (Send-closure variant audit), D267 (Family-2 Cat-3 Arc→Rc). Conflating them = framing error; cite this value when classifying cleanup candidates. Memory: `feedback_distinguish_vestigial_vs_speculative`. - -## Architectural invariants (compiler-enforced chokepoints) - -| Trait/Type | Signature | What it forbids | -|---|---|---| -| `CoreActor::run` | `F: FnOnce(&Core) -> R + Send + 'static` | Can't put `.await` in closure body (no async context) | -| `MailboxOp::Defer(SendDeferFn)` | `SendDeferFn = Box` | Deferred closure body must be sync; no `AsyncDefer` variant exists | -| `BindingBoundary::invoke_fn_with_core` | sync `fn(&self, .., core: &dyn CoreFull)` | Making binding callback async requires widening the trait → cross-track-ledger event | -| `Sink = Arc` (post-D248, !Send !Sync) | Sync `Fn` closure | Sinks can't be async; they fire synchronously during a wave | -| `Core::emit` / `Core::subscribe` etc. | All sync `pub fn` | Adding `pub async fn` in `graphrefly-core` would require tokio in Core — forbidden by D070/D077 | -| `Core: !Send + !Sync` (D248) | Move-only single-owner | Can't share `Arc` across threads; cross-Core parallelism is host-native via independent per-worker Cores | -| `BindingBoundary: Send + Sync` (FFI trait, unchanged by D248) | Send+Sync stays | Only subscriber callbacks (Sink/TopologySink/etc.) relaxed off Send+Sync; the FFI contract is unchanged | -| `CoreMailbox: Send + Sync` (D249) | Id-only ops + Send cross-thread Defer | DeferQueue is the !Send owner-only companion | - -## Invariant-watch list (red flags — HALT if any are proposed) - -1. `pub async fn` in `graphrefly-core` / `graphrefly-graph` / `graphrefly-operators` / `graphrefly-structures`. Only `graphrefly-storage` + `graphrefly-bindings-*` are sanctioned to import tokio. -2. New `MailboxOp::AsyncDefer(BoxFuture<...>)` variant. -3. Sink / TopologySink / NamespaceChangeSink becoming async return types. -4. Actor closure body calling `.await`. -5. napi method body chaining `.then(...)` or awaiting inside the actor closure (instead of inside the napi async fn wrapper). -6. `wrapper.js` stashing Promise chains in long-lived state. Promises resolve at the napi-call boundary; long-lived state = resolved values. -7. A reactive primitive returning `Promise>` in the binding layer. -8. Sync escape hatches on binding read methods (`run_sync` napi methods callable from inside TSFN sink callbacks). The bi-directional async-at-boundary invariant. -9. `BindingBoundary` widening without a cross-track-ledger row added FIRST. -10. Adding a new `Impl` parity-contract method without a parity scenario authored in the same slice. (D196 — "parity scenarios are the consumer pressure signal.") -11. Threadlocal-current-Core or similar machinery (`CURRENT_CORE`, `CoreThreadGuard`, `current_core()` accessor). D256 deleted these; re-adding them is regression. -12. `core.clone()` anywhere (`impl Clone for Core` was deleted at D221/D246). -13. Storing `Core` by value in a struct that's then put in `Arc<>` or `OnceLock<>` (Core is `!Send + !Sync` post-D248). -14. SchedulingGroupId speculative surface (D253 deleted it; re-adding without M6 consumer = regression). +| `~/src/graphrefly/decisions/decisions.jsonl` | **Unified D# log** (D1–D33 + DR-*). Canonical record: `{id, layer, question, decision, rationale, supersedes, status}`. | +| `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) | Full design narrative + 8 forced constraints + spec-amendment list + conformance hard scenarios. | +| `~/src/graphrefly/plan/antipatterns.jsonl` | Lessons / anti-patterns to flag against. | +| `~/src/graphrefly/spec/rules.jsonl` | Protocol rules — for "does the spec already pin this?". | +| Memory `feedback_*` files | The user's durable values/principles (below). | + +## Locked values (durable — cite by name) + +1. **No backward compat** (pre-1.0): structurally cleaner option, no legacy shims. `feedback_no_backward_compat`. +2. **No imperative triggers** in public API: reactive `ctx.up`/signals, not emitters/callbacks/timers+set; remove imperative paths when no caller depends. `feedback_no_imperative`. +3. **Single source of truth**: one canonical per concern; index points, never duplicates. `feedback_single_source_of_truth`. +4. **No autonomous decisions** (hard rule): surface spec↔code conflicts; don't silently pick; file-by-file review for multi-file rewrites. `feedback_no_autonomous_decisions`. +5. **No implement without approval**: decisions locked ≠ implementation approved. `feedback_no_implement_without_approval`. +6. **Verify premise before greenfield**: design tables lag code — grep symbols + check landed markers before a 9Q; stale premise = HALT. `feedback_verify_premise_before_greenfield`. +7. **Latest versions + context7** for current API docs. `feedback_latest_versions_context7`. +8. **Long-command observation discipline** (run-logged + DONE sentinel; no tail; no sleep-poll) and **subagent bg hygiene** (sync-verify or teardown before return). `feedback_long_command_observation`, `feedback_subagent_bg_hygiene`. + +## Clean-slate floor (never violate) + +**Sacred (L0.7):** topology declarative/serializable/inspectable · wave protocol is a public spec · +wave protocol impl is **sync** · all fn go through dispatcher. + +**Forced (F-*):** F-PERF (budget every abstraction) · F-PROTO-SPEC (spec+TLA++property) · +F-SYNC-CORE (dispatcher.invoke sync void) · F-DISPATCH-ALL (no inline-fn bypass) · +F-GRAPH-FIRST-API · F-NO-WEDGE-CUT (every primitive ≥2 segments) · +F-NO-IMPL-DEFINED (spec-locked or explicitly undefined-by-design) · F-NO-LLM-ONLY. + +**Red flags (HALT if proposed):** async in the sync wave core (async lives only in pools/wire-bridge) · +inline-fn bypassing dispatcher · a primitive serving only LLM workflows · a protocol behavior left +"implementation-defined" · user-replaceable onMessage/onSubscribe · adding a 10th tier casually · +graph-level shared mutable state accessed implicitly (must be explicit node + dep). ## Decision-process patterns (apply in order) -When asked "is X consistent / should I pick Y?": +1. **Identify the governing D#.** Grep `decisions.jsonl` by `layer`/keyword. Is the proposal within a locked D's scope? Mid-implementation scope expansion = anti-pattern unless promoted to a NEW D#. +2. **Check the spec.** Does `rules.jsonl` pin the behavior? If yes, follow it — divergence is a bug, not a design call. If silent/ambiguous → real design HALT. +3. **Verify premise (value 6).** Has the symbol/surface already landed? grep before designing new surface. +4. **Apply values + floor.** Especially: no autonomous decisions, no imperative, single source of truth, sync-core/async-at-boundary, F-* constraints. +5. **Verdict:** `consistent (cite D#)` / `regression on D#` / `out-of-scope` / `needs new decision (don't auto-pick)`. +6. **Routing:** + - New fork, no governing D# → present options + recommend, **do NOT lock** → that's `/design-review` → user approval → append `decisions.jsonl`. + - Changes protocol behavior → `/spec-amend` (spec-first). + - Cross-runtime parity concern → `/conformance` (behavioral scenario, not structural diff). -1. **Identify the locked decision-scope.** What D-number is this slice operating under? Is the proposed change within that D's locked scope? Scope expansion mid-implementation = anti-pattern unless promoted to a new D-number. -2. **Check the canonical spec.** Does the spec rule pin the behavior? If yes, follow spec — implementation that diverges is buggy, not a "design call." If spec is silent/ambiguous, that's a real design HALT. -3. **Check existing substrate surface.** Verify premise before greenfield: has the trait already been widened? Has the helper already been added? `feedback_verify_premise_before_greenfield` has paid off multiple times — always check before designing new surface. -4. **Apply the 8 user values above.** Especially: no autonomous decisions, no imperative triggers, single source of truth, sync-internal/async-at-boundary (bi-directional). -5. **Check D196 consumer pressure — BUT classify the candidate first.** Is the proposed change **new surface** (D196 applies; needs consumer signal) or **vestigial-surface removal** (D196 does NOT apply; decision-consistency restoration is the right frame — precedent D253/D254-AUDIT/D267, value #14)? Speculative-new widening fails D196; vestigial-cleanup is justified by the earlier relaxation that made the surface dead, no new consumer signal required. -6. **Check completeness AND discipline.** If the proposed change closes a real semantic gap, formalize as new D-number (Path X-via-D[N+1]). If the proposed change is speculative scope expansion, revert (Path Y trim). -7. **Identify cross-track-ledger events.** Does the change widen `Impl` parity contract? Does it cross presentation↔Rust-port? If yes, ledger row goes in BEFORE the change (`docs/cross-track-ledger.md`). -8. **Triage findings.** Match against `porting-deferred.md` (reject silently if matches) and `cross-language-notes.jsonl` `divergence-*` (reject silently if matches). Verified divergences ≠ hypothesized divergences — don't preemptively document the latter. - -## Common decision-shape templates - -### α/β/γ pattern (when napi binding shapes are forked — applies to S6+) - -- **α** — owner-thread-pinned `std::thread` worker + channel + oneshot reply -- **β** — mailbox-only async API (all ops post to mailbox + await reply) -- **γ** — sync owner-thread API via `spawn_blocking` onto pinned thread - -**Lock: α (D255).** β is dead permanently (D070/D077 libuv-busy deadlock recurrence — `bridge_sync` blocks libuv while waiting for TSFN microtask that needs libuv). γ collapses to α at impl level (`napi::tokio_runtime::spawn_blocking` has no thread affinity; `LocalSet` is invasive; γ.ii = std::thread + channel + oneshot IS α). - -### A/B/C pattern (when fix shape is forked) - -Default: the option that **structurally extends an existing pattern** wins over per-site workarounds. Recent applications: -- **D260** (wave-end re-drain loop) — A extends D232-AMEND/D249's drain-to-quiescence past `fire_deferred`. B re-introduces D256-deleted threadlocal. C per-sink eager nested waves change mid-batch ordering without principled rationale. → A wins. -- **F2 S6 fix** (RefCell reentrant borrow panic) — B restructure with RAII Guard is more general; C specialized helper is minimal diff. Either valid; lean B for long-term API generality, C for short-term scope. - -### Path X / Path Y / Path Z pattern (when slice scope is contested) - -- **Path X-direct** — continue under original D, fix bug introduced by autonomous scope expansion. **Anti-pattern.** -- **Path X-via-D[N+1]** — stop tracing under original D; lock new semantic as new D-number; resume under new banner. **Right pattern when scope expansion catches a real semantic gap.** -- **Path Y** — revert scope expansion; ship original D's locked scope only. **Right pattern when scope expansion is speculative.** -- **Path Z** — defer entire slice; redesign later. **Right pattern when neither shipping nor continuing is principled.** - -### Adversarial-review pattern (for /qa) - -- **Critical** — show-stopper, breaks first call. Fix or hard HALT. -- **Major (needs decision)** — architecture-affecting or ambiguous fix. User decides between options. -- **Auto-applicable** — clear fix following existing patterns. Batch with approval. -- **Reject** — false positive, matches a porting-deferred entry, or matches a `divergence-*` entry. Drop silently. - -## Locked decisions index (concise; load `rust-port-decisions.md` for full) - -| D# | Scope | One-line | -|---|---|---| -| D246 | β-simplification lock | Single-owner pushed all the way down; delete shared-Core machinery; ignore legacy ergonomics. Operating rule: NO stubs/deferrals — finish each S fully. | -| D247 | S2c Graph tree shape | `Rc>` (not owned-`&mut`); Graph becomes `!Send+!Sync`. | -| D248 | S2c Sink contract relax | Substrate `Sink`/`TopologySink` dropped `Send+Sync` → `Core`/`OwnedCore`/`Graph` now `!Send + !Sync`. `BindingBoundary` stays `Send+Sync` (FFI). | -| D249 | S2c Defer/mailbox split | Minimal owner-only `!Send` `DeferQueue` split off `Send` `CoreMailbox`. `Core::drain_mailbox` drains BOTH queues to mutual quiescence. | -| D250 | S4 retire imperative re-entry stubs | 3 pause/resume/set_deps in-wave re-entry tests retired as deleted-model (synchronous binding-clones-Core trigger is structurally gone). NO new substrate surface. | -| D251 | S4 rule-8 reusable coalescing slot | Per-handle `Cell` scheduled gate + observe-prune torn-id buffer in observe/describe/storage defer paths. One Box + one snapshot per wave instead of per emission. Internal refactor. | -| D252 | S5 IN_TICK collapse | `IN_TICK_OWNED: AHashSet` → `Cell`. Hard invariant: "one Core per OS thread, no nested cross-Core driving on a single thread." Panic-on-violation at BatchGuard claim. | -| D253 | S5 SchedulingGroupId delete | D196-pure deletion of `SchedulingGroupId` + `node_group` + `partition_of`/`group_of`/`set_scheduling_group` API. Reverses S3 rename. Re-introduce when M6 consumer materializes. | -| D254 | S5 Tier A bundle | DeferQueue Mutex→RefCell + AtomicBool→Cell; test-file doc sweep; process-discipline memory. **D254-AUDIT inline**: TimerEmit typed variant audit found ZERO surviving emit-only Send closures; not adopted. | -| D255 | S6 napi binding shape | α/γ-merged actor model (β dead, γ collapses to α). `crates/graphrefly-bindings-js/src/core_actor.rs` owns Core on dedicated worker thread. | -| D256 | S6 invoke_fn_with_core override | Premise-check win: D245 already added the surface. BenchBinding overrides `BindingBoundary::invoke_fn_with_core`; `CURRENT_CORE` thread-local + `CoreThreadGuard` + `current_core()` accessor **deleted outright**. No cross-track-ledger event (consumption-not-widening). | -| D257 | S6 Drop discipline | BenchCore::Drop dispatches detached unsubscribe via actor with try_send fallback. | -| D258 | S6 WORKER_EXTRAS | Owner-thread-only `thread_local!(RefCell>>)` for `!Send` resources (Graph, StorageHandle, LogView/ScanHandle/ReactiveSub). | -| D260 | S7 wave-end re-drain loop | `BatchGuard::Drop` extends drain past `fire_deferred` to absorb posts made by deferred-jobs themselves. Mutual quiescence in `mailbox + DeferQueue + fire_deferred`. Same `max_batch_drain_iterations` cap. | -| D261 | S7 try_subscribe tail drain | Brief BatchGuard at end of `try_subscribe` reuses D260's drain machinery to flush handshake-fire-time posts. | -| D266 | Family-1 sink Arc→Rc cleanup | Decision-consistency restoration post-D248 sink Send+Sync relaxation. 4 type aliases (Sink/TopologySink/DescribeSink/NamespaceChangeSink) Arc→Rc + operator-internal !Send !Sync `Arc` callback fields + 4 `static_assertions::assert_not_impl_any!(...: Send, Sync)`. 13 per-file `#![allow]` annotations removed. No `Impl` widening. Sequenced after D265/F1. | -| D267 | Family-2 Cat-3 `Arc>` → `Rc>` cleanup | Decision-consistency restoration; compiler-driven sweep. Try the substitution workspace-wide; compiler classifies Category-1 (bindings Send+Sync required) and Category-2 (storage Send+Sync required) via fail-to-compile → revert; residual compiling set IS Category-3 (operator-internal + test-recording, single-owner). Per-revert one-line comment naming Send+Sync source. Mutex `lock().unwrap()` → RefCell `borrow_mut()`. | -| D268 | Vestigial union-find / defer-shim surface cleanup | Decision-consistency restoration post-D248/D253/D255 relaxation chain. Family A: delete `PartitionOrderViolation` struct + `SubscribeError::PartitionOrderViolation` variant + 9 dead `Err(_)` match arms + 7 `Result<(), PartitionOrderViolation>` fn signatures. Family B: delete `emit_or_defer`/`complete_or_defer`/`error_or_defer` from Core + BindingBoundary trait + CoreFull impl + default impls; delete `DeferredProducerOp` enum + `push_deferred_producer_op` + `drain_deferred_producer_ops` no-op shim; edit ~15 operator call-sites in `buffer.rs`. Sub-option A.i: inline `try_emit/try_complete/try_error` (pub(crate)) bodies into public `emit/complete/error` — pre-flight confirmed ZERO external callers. Net ~110 LOC, mostly deletions. | - -**M1 (QA /qa 2026-05-19)** — Cross-queue FIFO inversion documented as new contract: CoreMailbox drains before DeferQueue every round (queue-priority); intra-queue FIFO preserved. Regression test in `lock_discipline.rs::cross_queue_order_mailbox_then_deferred`. - -**M2 (QA /qa 2026-05-19)** — `compact_every` cadence restored to per-emission count (TS parity). `pending_count` tracks qualifying emits at filter gate; `flush_tier(s, snapshot, count)` advances `flush_count` by count. - -**Decision-audit batch (2026-05-21)** — D266/D267/D268 locked together as a decision-consistency cleanup train; bundled with doc-hygiene + 3 AMEND-D edits into one /porting-to-rs run sequenced after D265/F1 (parallel session). Doc-hygiene: L4-001 (Core rustdoc) + L4-002 (GraphOps doclinks) + L8-001 (close §7-E/§7-B/§7-F as resolved by D253/D255) + L8-002 (D250 stub-deletion history collapse) + L8-003 (CoreShared/StateCell references) + L1-001 (MailboxOp::Defer rustdoc). AMEND-D: D262/P4 affects-list + D267 wording scope precision (factory constructors retain `run_sync` under lifecycle-precondition rationale; absorbs the 4 `create` factory finding) + porting-deferred §7-C framing (decision-consistency restoration, not D196 deviation; closes when D268 lands). Audit doc: `~/src/graphrefly-rs/docs/decision-audit-2026-05-21.md`. Full L2 invariant-watch sweep #1-#14 CLEAN; L7 TRASH/ confirmed non-compiled. - -## Pending / open decisions (track but not yet locked at time of skill creation) - -These were under active discussion in the session this skill was created from; current state may have moved by next session — check `rust-port-decisions.md` first. - -- **D262** — `compact_every` per-emission count (M2 from /qa). Likely locked by now. -- **D263** — `terminal_as_real_input` gate-predicate + flag-surfacing. Original framing: "no semantic change; predicate already conformant; just surface flag." Chat extended scope mid-implementation with `skips_auto_cascade`. **Recommendation: Path X-via-D264** (formalize the auto-cascade-skip semantic as its own D-number if it's the right completion of `terminal_as_real_input`'s meaning). -- **D264 (proposed)** — `terminal_as_real_input` complete semantic with `skips_auto_cascade`, IF spec-cited. Needs spec citation from canonical-spec §5.4 / §519 / R5.4 to confirm auto-cascade-skip is in the spec's text. -- **D265 (proposed)** — graph_bindings F1 fix: convert sync read methods (`nameOf`, `tryResolve`, `nodeCount`, etc.) to `async fn` to eliminate D070/D077 sink-callback re-entrance deadlock. Refine `Impl` parity contract to `T | Promise` so pure-ts arm doesn't artificially wrap sync reads. Cross-track-ledger row. - -D263 /qa findings (pending lock): -- **D1.a** — drop `|| self.partial` from `skips_auto_cascade`. Don't OR orthogonal design dimensions. Test rewrite ~10 LOC. -- **D2.a** — `addDep` always calls Core (no JS-side preflight). Single source of truth. -- **D3** — REJECT (canonical-shape consistency with `fire_operator`); mandatory doc note on `register_user_derived` covering NO_HANDLE deps[i] surface. - -## Remaining work (post-S5/S6/S7) - -- **S6 follow-on bench TODOs** (per-call latency, channel backpressure, multi-instance OS-thread count) — measurement-driven, not blocking. -- **D265 / F1 fix** for graph_bindings sync-read-method deadlock. -- **Storage-parity follow-up** (cross-track-ledger §2: appendLogStorage flush() durability + reject + rollback epoch). -- **attach_storage re-ship-on-shrink** (pre-existing structures-storage smell). -- **Loom verification** — outside the gate; periodic check. -- **Native publish** (D203/D204 — human tag-push gate). -- **M6** — pyo3 + per-binding pluggable group executor (post-1.0). -- **D080** — async-everywhere presentation rebase (deferred until consumer pressure). - -## Recurring anti-patterns to call out - -1. **Autonomous scope expansion** — chat extends a locked D's scope with a new behavior, hits a bug, asks permission to debug. Right answer: **stop. Formalize as new D-number with proper design + test plan, then resume under new banner.** Don't trace bugs under wrong-D. -2. **"Completeness" used to justify autonomous expansion** — when chat argues "Path X gives more complete results," check if completeness is real (genuine spec/semantic gap) or speculative (chat extending without consumer pressure). If real, formalize as new D. If speculative, revert. -3. **Stale-premise propagation** — chat builds a recommendation on a premise that the substrate has moved past (D245 had already added the surface chat was about to recreate). Verify premise via grep / canonical doc / decision log before greenlighting. -4. **Hypothesized divergence preemptive-doc** — adding a `cross-track-ledger.md` or `cross-language-notes.jsonl` entry for a divergence that hasn't been verified by a cross-arm parity test. **Write the test first; document only verified divergences.** -5. **Dual-source-of-truth fixes** — JS-side preflight mirroring Core invariants; binding-side snapshot kept in sync with actor state. All such fixes drift; reject in favor of single-source-of-truth routing through the authority. -6. **Sync escape hatch on binding reads** — `run_sync` napi methods callable from inside TSFN sink callbacks. Violates bi-directional async-at-boundary. All read methods must be `async fn`. -7. **Test expectations from intuition** — writing `expected = [1, 2]` instead of citing R1.3.5.a + the push-on-subscribe handshake rule. Always anchor test expected vectors in spec R-IDs. -8. **OR-ing orthogonal design flags** — `skips_auto_cascade = self.terminal_as_real_input || self.partial`. Collapses two design dimensions into one; loses expressivity; widens one flag's semantic silently. Each flag's semantic is locked independently; users compose them explicitly. -9. **Preemptive skip-markers** — `runIf(impl.name !== "rust-via-napi")` added preemptively before a divergence is verified. You skip tests with a consumer-driven reason, not preemptively. -10. **Deferring documentation** — comment-update sweeps deferred from S[N] to S[N+1] create reviewer noise + risk of missing sites. Delete-the-code-and-its-docs in the same commit. -11. **D196 misapplication on vestigial-surface cleanup** — waving D196 ("no speculative substrate") at a cleanup that's actually restoring decision-consistency post-relaxation (Family-2 Cat-3 / D267 was the canonical instance; D266→D267 framing was initially wrong). Vestigial surface ≠ speculative surface. When triaging a deferred cleanup item, ask: was this surface made dead by an earlier locked relaxation (D246/D247/D248/D249/D252/D253/D256/...)? If yes, the gate is **decision-consistency restoration** (precedent D253/D254-AUDIT/D267), not D196. Value #14 covers this; cite by name. -12. **"Orthogonal" sub-decisions whose orthogonality wasn't tested** — D301's B.a (drop reserved-prefix guard) and B.b (keep `_anon_` snapshot marker) were framed as orthogonal during the Q4 sub-decision lock. They weren't: B.a let users register `_anon_42` as a node name, and B.b emitted `_anon_42` for unresolvable cross-mount deps with `NodeId(42)`. The `SnapshotError::UnresolvableDeps` Debug-format diagnostic could no longer distinguish "user-named node `_anon_42` failed to hydrate" from "anonymous dep with NodeId 42 couldn't be resolved" — collision caught only at /qa, not at design lock. **When sub-decisions are framed as orthogonal, sketch one input that exercises BOTH simultaneously — confirm the orthogonality survives the example before locking.** The B.a/B.b case took ~30 seconds: "user registers a node named `_anon_42`; snapshot encodes an unresolvable dep with NodeId(42); what does the diagnostic say?" — if that one-input sketch had been part of the lock checklist, the coupling would have surfaced at design-time. Add this sketch as a mandatory pre-lock micro-check for any multi-sub-decision question. Source: D301 /qa (2026-05-26). - -## How to use this skill in a new session - -When the user asks any of the trigger phrases: - -1. **Invoke decision-guard skill** to load this context. -2. **Read the relevant section** for the question type (values for "is this consistent," shapes for "should I pick A/B/C," anti-patterns for "what about this chat proposal"). -3. **Check the locked decision index** for prior D-numbers that bear on the question; load `rust-port-decisions.md` excerpt if the full text is needed. -4. **Cross-check `porting-deferred.md`** for already-acknowledged deferrals (reject-silently for /qa findings; spec-extension for new ones). -5. **Apply the decision-process pattern** (the 8 ordered steps above). -6. **Produce a relay-ready summary** for the user to paste back into the chat that surfaced the question. The summary should: - - Cite the relevant decision/value/invariant by name. - - Give the recommended pick (A/B/C, X/Y/Z, α/β/γ). - - Explain the reasoning in 2-3 sentences per option. - - Flag any HALT-worthy concerns (autonomous expansion, stale premise, dual-source-of-truth, etc.). - -## Skill scope boundaries - -This skill is **read-mostly**: it loads decisions + values + patterns. It does NOT: -- Run gates (`mise run gate`) — that's `/porting-to-rs` or `/qa`. -- Apply fixes — that's `/dev-dispatch` or `/qa` post-decision. -- Author parity scenarios — that's a follow-up `/dev-dispatch` after a decision locks. -- Replace `/porting-to-rs` HALTs — those have their own Phase 1/2 protocol. - -Invoke decision-guard when the question is **"what should I decide?"**, not "what should I do?". The output is decision + reasoning + relay-ready text. Implementation follows separately. +## Common decision shapes -## Update protocol +- **A/B/C (fix shape):** default = the option that **structurally extends an existing pattern** beats per-site workarounds. +- **Completeness vs discipline:** if a proposal closes a real semantic gap → formalize as a NEW D# (don't continue under the original D's banner). If speculative scope expansion → revert. +- **Orthogonal sub-decisions:** before locking two "orthogonal" sub-decisions, sketch ONE input exercising BOTH — confirm orthogonality survives the example (antipatterns.jsonl; the 30-second check that catches coupling at design-time). -When a new D-number locks (after user approval), append to: -- `~/src/graphrefly-ts/docs/rust-port-decisions.md` — full entry (Date / Context / Options / Decision / Rationale / Affects) -- This skill's "Locked decisions index" — one-line summary -- `~/src/graphrefly-rs/docs/migration-status.md` — if the lock closes/scopes a slice +## Scope boundaries -When a new user value surfaces (in feedback memory format), add to: -- `~/.claude/projects/-Users-davidchenallio-src-graphrefly-ts/memory/feedback_.md` -- This skill's "User values & principles" section — pointer + one-line summary +Read-mostly. Loads decisions + values + patterns; produces **decision + reasoning + relay-ready text**. +Does NOT run gates, apply fixes, or author scenarios — those are `/dev-dispatch` / `/qa` / `/conformance` +after a decision locks. Invoke when the question is "what should I decide?", not "what should I do?". -When an anti-pattern recurs (caught a 2nd time in /qa or design), add to: -- This skill's "Recurring anti-patterns to call out" section -- Optionally a `feedback_.md` memory if it's a generalizable principle. +## Update protocol -Keep this skill **tight**: aim for ≤ ~500 lines of skill markdown. If it grows past that, factor sub-skills (`decision-guard:invariants`, `decision-guard:decisions`) and have the main skill point at them. +When a new D# locks (after user approval): append to `~/src/graphrefly/decisions/decisions.jsonl` +(`{id, layer, date, question, decision, rationale, supersedes, status:"locked", session}`), update the +session's `locks` in `sessions.jsonl`, and run `node ~/src/graphrefly/dashboard/build.mjs --check`. +When a new anti-pattern recurs: append to `~/src/graphrefly/plan/antipatterns.jsonl` (+ a `feedback_*` +memory if generalizable). When a new durable value surfaces: add a `feedback_.md` memory + a +pointer line here. diff --git a/.claude/skills/parity/SKILL.md b/.claude/skills/parity/SKILL.md deleted file mode 100644 index a07c5a96..00000000 --- a/.claude/skills/parity/SKILL.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -name: parity -description: "Cross-language parity check + adversarial QA pass across graphrefly-ts and graphrefly-py. Run after /dev-dispatch + /qa on both repos. Use when user says 'parity', 'cross-lang check', or 'sync repos'." -disable-model-invocation: true -argument-hint: "[feature area or 'full'] [optional: path to sibling repo]" ---- - -You are executing the **parity** workflow, comparing **graphrefly-ts** (this repo) against **graphrefly-py** (`~/src/graphrefly-py` unless overridden in $ARGUMENTS). - -**This repo is the single source of truth** for all operational docs (roadmap, optimizations, test-guidance, docs-guidance, archive). Both repos' docs are maintained here. - -Context from user: $ARGUMENTS - ---- - -## Phase 1: Scope & Gather - -Determine scope from $ARGUMENTS: -- If a **feature area** is given (e.g. "Graph 1.3", "batch", "node lifecycle"), focus on that area only. -- If `full`, scan all implemented phases in both repos. - -> **PY parity is currently PARKED until 1.0** per the 2026-04-30 re-prioritization in `docs/implementation-plan.md` § Parked. Run `/parity` only when explicitly invoked by the user (e.g. for a one-off audit), or when post-1.0 work resumes. Findings during the parked window get filed to `optimizations.md` under a `[py-parity-*]` tag and are NOT scheduled for implementation until the umbrella reopens. - -Read in parallel: -- **Operational docs (this repo):** `docs/implementation-plan.md` (canonical pre-1.0 sequencer; the matching phase tells you what's locked vs in-flight), `docs/optimizations.md` (active items + deferred, line-item state for `[py-parity-*]` carries), `archive/optimizations/*.jsonl` (cross-language notes, resolved decisions — search with `grep`), `docs/roadmap.md` (vision context only; do NOT use as the sequencer), `~/src/graphrefly/GRAPHREFLY-SPEC.md` (relevant sections) -- **Composition guide:** `~/src/graphrefly/COMPOSITION-GUIDE.md` — **mandatory** when the scoped area includes `packages/pure-ts/src/patterns/` or `packages/pure-ts/src/compat/` in either repo. Composed factories require understanding lazy activation, subscription ordering, null guards, wiring order, feedback cycles, and SENTINEL gate patterns. -- **TS source:** `packages/pure-ts/src/` and `packages/pure-ts/src/__tests__/` in the scoped area -- **PY source:** `~/src/graphrefly-py/src/graphrefly/` and `~/src/graphrefly-py/tests/` in the scoped area - -**Important:** Read `archive/optimizations/cross-language-notes.jsonl` entries with `id` prefix `divergence-`. These are **confirmed intentional divergences** — do NOT re-raise them as parity gaps or QA findings. Filter them out before presenting results. - ---- - -## Phase 2: Diff — API Surface - -For the scoped area, compare: - -1. **Public API** — method names, signatures, options/kwargs, return types -2. **Error behavior** — what throws/raises, error messages, error types -3. **Edge cases** — boundary conditions, validation rules (e.g. name constraints, duplicate handling) -4. **Default behavior** — what happens when optional args are omitted -5. **Subpath tier (TS-only; informational for PY)** — for each symbol, note its TS tier: universal (browser + Node safe), `/node` (Node-only), or `/browser` (DOM-only). See `docs/docs-guidance.md` § "Browser / Node / Universal split". Record this column even though PY has no equivalent split yet — when PY adds a comparable feature (e.g. `fileStorage`), the decision should match TS (filesystem I/O goes in a Node-only module). Treat tier mismatches where PY exposes something from a "universal"-shaped module that TS places under `/node` as a flag for future PY-side consideration, not as a blocking divergence today. - -Present a table: - -| Aspect | TypeScript | Python | Verdict | -|--------|-----------|--------|---------| -| ... | ... | ... | aligned / TS ahead / Py ahead / intentional divergence | - -Mark **intentional divergences** (language idiom, concurrency model, etc.) separately from **unintentional gaps**. - ---- - -## Phase 3: Diff — Behavioral Semantics - -For unintentional gaps found in Phase 2, dig deeper: - -1. Read the **implementation** on both sides -2. Read the **tests** on both sides -3. Identify the **spec-correct** behavior per `~/src/graphrefly/GRAPHREFLY-SPEC.md` -4. For items not covered by the spec, check `docs/optimizations.md` open design decisions - -For each gap, classify: -- **spec-decided** — spec says what the behavior should be; one side is wrong -- **convention-decided** — `optimizations.md` cross-language notes already aligned on this -- **needs-decision** — neither spec nor optimizations.md covers this; flag for user - ---- - -## Phase 4: Cross-Repo Adversarial QA - -This is a **second QA pass** that catches issues the per-repo `/qa` missed — bugs that only surface when you read both implementations side by side. - -### 4a. Gather both diffs - -Run `git diff` in **both** repos. Also read any untracked files in the scoped area. - -### 4b. Launch parallel review subagents - -Each subagent receives the diffs from **both** repos plus the cross-language notes from `docs/optimizations.md`. - -**Subagent 1: Parity Semantic Hunter** — Has read access to both repos: -> You are a Parity Semantic Hunter reviewing **two implementations** of the same reactive graph protocol (graphrefly-ts and graphrefly-py). Both repos just had independent `/dev-dispatch` + `/qa` runs. First, read `archive/optimizations/cross-language-notes.jsonl` and collect all entries with `id` prefix `divergence-` — these are **confirmed intentional divergences** that must NOT be raised as findings. Then review the diffs side by side for: message ordering mismatches between ports, settlement/batch timing differences, edge cases where one port handles a scenario the other doesn't, validation rules present in one but missing from the other, test coverage asymmetry (scenario tested on one side but not the other), naming or path convention drift. For each finding: **title** | **severity** (critical/major/minor) | **which repo** | **detail** | **suggested fix**. - -**Subagent 2: Spec Conformance Hunter** — Has read access to both repos + spec: -> You are a Spec Conformance Hunter. Read `~/src/graphrefly/GRAPHREFLY-SPEC.md` and both diffs. First, read `archive/optimizations/cross-language-notes.jsonl` and collect all entries with `id` prefix `divergence-` — these are **confirmed intentional divergences** that must NOT be raised as findings. Check whether either implementation drifted from the spec during implementation: incorrect message ordering, wrong terminal behavior, batch semantics that don't match spec §2, node lifecycle violations, graph composition contracts (§3) not met, `describe`/`observe` output that doesn't match Appendix B. Also check design invariant violations (spec §5.8–5.12): polling patterns, imperative triggers bypassing graph topology, raw async primitives (Promises/microtasks in TS, asyncio.ensure_future/create_task in PY) for reactive scheduling, direct time API usage instead of central clock, hardcoded message type checks instead of messageTier/message_tier, and Phase 4+ APIs leaking protocol internals. Also check whether `docs/optimizations.md` cross-language decisions are actually implemented correctly on both sides. For each finding: **title** | **severity** | **spec section** | **which repo(s)** | **detail**. - -### 4c. Triage QA findings - -Classify each finding: -- **patch** — fixable code issue; include which repo needs the fix -- **defer** — pre-existing, not caused by this round of changes -- **reject** — false positive - ---- - -## Phase 5: Present Findings (HALT) - -Present ALL findings from Phase 2–4 to the user, grouped: - -### Group 1: Parity Gaps — Auto-fixable -For each: the gap, which repo needs the fix, the fix description, effort (S/M/L). - -### Group 2: QA Findings — Auto-fixable -For each: the issue, which repo, the fix, effort (S/M/L). - -### Group 3: Needs Decision -For each: the gap or issue, both behaviors, spec/convention silence, recommended resolution. - -### Group 4: Intentional Divergences (FYI) -Language-specific differences correct on both sides (thread safety, `|` operator, etc.). - -**Wait for user approval before proceeding.** - ---- - -## Phase 6: Apply Fixes - -After user approves: - -1. Apply fixes to **this repo** (graphrefly-ts) — code + tests -2. Run `pnpm test` — fix failures -3. If fixes approved for the **sibling repo**, apply those too: - - Code + tests in `~/src/graphrefly-py/` - - Run `cd ~/src/graphrefly-py && uv run pytest` — fix failures -4. Update `docs/optimizations.md` (this repo — single source of truth for both): - - Add new open decisions under "Active work items" (line-item state for any new `[py-parity-*]` carry). - - **Actively sweep:** scan for any fully-resolved items (all sub-tasks DONE, no remaining TODOs) and archive them to `archive/optimizations/resolved-decisions.jsonl` per `docs/docs-guidance.md` § "Optimization decision log". Remove archived content from `optimizations.md`. -5. Update `docs/implementation-plan.md` (canonical sequencer): - - When a `[py-parity-*]` item lands or its scope shifts, mark it ✅ in the matching phase entry (or note "PY parity carry" within the relevant Phase 11–16 sub-section). When PY parity reopens post-1.0, the phase entry is where future agents pick up scope. - - If a parity pass closes the **last in-flight item from a Phase**, archive the phase body to `archive/roadmap/phase--*.jsonl` and replace with a 2–4-line summary + archive pointer per `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/implementation-plan.md`". Single residual follow-ups move to `optimizations.md` with a back-link. - - Do NOT add new sequencing here during the parked window; just record state changes. -6. `docs/roadmap.md` is **vision context only** per 2026-04-30 migration — do NOT track item-level state here. Wave-completion archival to `archive/roadmap/*.jsonl` (with a one-line pointer left behind) still applies per `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/roadmap.md`" but rarely fires during /parity. - ---- - -## Phase 7: Final Checks - -Run all checks on both repos and fix any failures: - -**TypeScript:** -```bash -pnpm test && pnpm run lint:fix && pnpm run build -``` - -`pnpm run build` runs `assertBrowserSafeBundles` post-build. If it fails, a TS change leaked a Node builtin into a universal entry — fix per `docs/docs-guidance.md` § "Browser / Node / Universal split" before closing the parity pass. - -**Python:** -```bash -cd ~/src/graphrefly-py && uv run pytest && uv run ruff check --fix src/ tests/ && uv run mypy src/ -``` - -Report results. If any failures relate to a design question, HALT. diff --git a/.claude/skills/porting-to-rs/SKILL.md b/.claude/skills/porting-to-rs/SKILL.md deleted file mode 100644 index ea481045..00000000 --- a/.claude/skills/porting-to-rs/SKILL.md +++ /dev/null @@ -1,326 +0,0 @@ ---- -name: porting-to-rs -description: "Port a slice of GraphReFly from TS to Rust (graphrefly-rs). Use when user says 'port to rust', 'porting-to-rs', or provides a task to add/extend functionality in the Rust workspace. Mirrors /dev-dispatch's plan→halt→implement→self-test loop, specialized for cross-repo Rust port work. Run /qa afterward for adversarial review." -argument-hint: "[--light] [task description or context]" ---- - -You are executing the **porting-to-rs** workflow — implementing or extending a slice of the **GraphReFly** Rust port (`graphrefly-rs`, `~/src/graphrefly-rs`) against the canonical TS spec living in `graphrefly-ts` (this repo). - -This skill is the Rust-port counterpart to `/dev-dispatch`. Same plan→halt→implement→self-test shape, but the canonical authority and invariants are different: the Rust port targets the **post-Phase 13.6.A consolidated canonical spec** (single document), not the multi-file TS spec + composition guides. - -The user's task/context is: $ARGUMENTS - -### Mode detection - -If `$ARGUMENTS` contains `--light`, this is **light mode** — skip Phase 2 HALT unless escalation triggers (see Phase 2 below). Otherwise, this is **full mode** with mandatory architecture discussion before implementation. - ---- - -## Phase 1: Context & Planning - -Load context and plan the implementation in a single pass. **Parallelize all reads.** - -### Canonical authorities (READ FIRST) - -These supersede / consolidate the multi-file TS authority for Rust port purposes: - -- **`docs/implementation-plan-13.6-canonical-spec.md`** — *single-document* canonical spec for the Rust port. Folds `~/src/graphrefly/GRAPHREFLY-SPEC.md` + all four `COMPOSITION-GUIDE-*.md` files into one read-once handoff. **This is THE behavior authority for the Rust impl.** Use the rule-ID convention (`R
.[.letter]`) for cross-references in commits, comments, test names. Sections of interest: - - §1 Message Protocol — tier table (R1.3.7.b), message variants, payload-handle discipline - - §2 Node — lifecycle (R2.2.7 resubscribable, R2.5.3 first-run gate, R2.6 PAUSE/RESUME, R2.6.4 TEARDOWN-precedes-COMPLETE) - - §3 Graph — container, mount/unmount, sugar - - §5 Design Principles (R5.1–R5.12) - - §6 Implementation Guidance — explicit TS / PY / Rust deltas - - §11 Implementation Deltas — known TS-vs-canonical-spec drift; the Rust port targets the canonical, NOT the current TS code -- **`docs/implementation-plan-13.6-flowcharts.md`** — Mermaid diagrams visualizing every internal method, property, and process referenced by the canonical spec. Cross-referenced via rule IDs. Use when: - - A spec rule needs to be implemented and you need to see the call/data flow shape - - A red 🟥 node flags TS-vs-canonical drift (the Rust port should match canonical) - - A yellow 🟨 node flags not-yet-implemented features (out-of-scope for current slice unless explicitly part of $ARGUMENTS) - - **Especially Batch 7 — Rewire substrate** (Phase 13.8; experimental TS impl mirrored in `graphrefly-rs/crates/graphrefly-core/tests/setdeps.rs`) - -### Rust port operational docs (READ NEXT) - -- **`~/src/graphrefly-rs/docs/migration-status.md`** — **canonical milestone tracker** for the 6-milestone Rust port. Read FIRST to know: - - What landed (M1 dispatcher, M1 parity Slice A+B, etc.) - - What's blocked / in-progress - - The closing section format (each closed milestone gets a `## M — closed YYYY-MM-DD` block summarizing what landed, what was deferred, and how it maps back to the migration plan) - - The "Carried forward" pointer to porting-deferred.md -- **`~/src/graphrefly-rs/docs/porting-deferred.md`** — running registry of audit-surfaced concerns deferred to evidence-driven slices. Read to know: - - Which §10 perf simplifications are deliberately deferred (and why — Pass 5 bench evidence) - - v1 dispatcher limitations (re-entrance, sink-fire lock discipline, recursion stack overflow, etc.) - - Spec divergences acknowledged in v1 (e.g., pause-buffer overflow not synthesizing ERROR; alloc_lock_id collision risk) - - Open questions from `archive/docs/SESSION-rust-port-architecture.md` Part 6 - - Audit fixes that landed (so we don't re-introduce them) - -### Cross-repo context (READ AS NEEDED) - -- **`archive/docs/SESSION-rust-port-architecture.md`** — the migration plan: 6-milestone phasing, layer-by-layer port recommendation, deferral guardrails. Read FIRST when picking up port work in a new area. -- **`docs/research/handle_protocol.tla` + `handle_protocol_MC.tla`** — TLA+ refinement of `wave_protocol.tla` over the handle abstraction. The Rust port must satisfy the same invariants. -- **`docs/research/handle-protocol-audit-input.md`** — per-rule classification (which 13.6 invariants are Core-internal vs binding-layer). Use as the layer-classification key during M1–M5. -- **`packages/pure-ts/src/__experiments__/handle-core/core.ts` + `bindings.ts`** — TS prototype reference impl (~370 lines each, 22 invariant tests). The Rust port mirrors this module-for-module for the M1 dispatcher slice. (Post-Phase-13.9.A cleave: the pure-TS impl moved from root `src/` to `packages/pure-ts/src/`. The root `src/` is now the `@graphrefly/graphrefly` shim — re-exports only, no logic.) -- **`packages/pure-ts/src/core/node.ts` + supporting files** — TS production dispatcher. Reference for parity behavior, NOT for code structure (the Rust port follows the canonical spec, not the current TS shape — see §11 Implementation Deltas). -- **`packages/parity-tests/`** — cross-impl parity scenarios (Phase 13.9.A). When a Rust slice closes a milestone listed in `packages/parity-tests/README.md` schedule (M1 dispatcher, M2 Slice E Graph, M3 operators, M4 storage, M5 structures), the slice should ALSO add corresponding scenarios to `packages/parity-tests/scenarios//.test.ts` parameterized via `describe.each(impls)`. The `rustImpl` arm currently exports `null` and activates when `@graphrefly/native` (the napi binding) publishes its package shape — until then, scenarios run against `pureTsImpl` only but the structural parameterization is in place. -- **`docs/implementation-plan.md`** Phase 13.7 / 13.8 / 13.9 — Rust M1 bench feasibility study + TS rewire integration tests + the Phase 13.9.A cleave. Cross-reference for what bench data exists / what's been tested in TS / how the cleaved package architecture works. - -### Rust workspace layout - -``` -~/src/graphrefly-rs/ -├── crates/graphrefly-core/ # M1: dispatcher, message tiers, batch, wave engine -├── crates/graphrefly-graph/ # M2: Graph container, snapshot, content addressing -├── crates/graphrefly-operators/ # M3: built-in operator types -├── crates/graphrefly-storage/ # M4: tier dispatch + Node-only persistence (redb) -├── crates/graphrefly-structures/ # M5: reactiveMap/List/Log/Index (imbl) -├── crates/graphrefly-bindings-js/ # M1+: napi-rs JS bindings -├── crates/graphrefly-bindings-py/ # M6: pyo3 Python bindings -└── crates/graphrefly-bindings-wasm/ # WASM target -``` - -`cargo build` / `cargo nextest run` (no `--workspace`) excludes the bindings crates by default — `default-members` skips them since they need their own toolchains (napi-rs, maturin, wasm-pack). **Use `cargo-nextest`, not legacy `cargo test`:** `cargo nextest run -p graphrefly-core` for the typical Rust-only inner loop (the `cascade_depth` stack-safety stress tests are quarantined from the default profile — see `graphrefly-rs/.config/nextest.toml`); `cargo nextest run --profile ci` (alias `cargo tc`) for the full merge-gating suite incl. those guards; `scripts/dev-test.sh` to run the default loop with a per-worktree-isolated target dir (parallel sessions never share the `target/` build lock). Legacy `cargo test` is fallback-only and ignores the nextest profiles + slow-timeout hang-kill. Loom is the one exception — it stays on `cargo test -p graphrefly-core --features loom-checked`. - -### Reads to perform in parallel - -- `~/src/graphrefly-ts/docs/implementation-plan-13.6-canonical-spec.md` (deep-read sections relevant to $ARGUMENTS) -- `~/src/graphrefly-ts/docs/implementation-plan-13.6-flowcharts.md` (find the batch matching the slice) -- `~/src/graphrefly-ts/docs/cross-track-ledger.md` (every time, no exceptions — **surface any unlanded native-side rows as candidates with `[NEEDS-LOCK]` markers for un-D-numbered items**; this is the auto-intake that prevents ledger rot, e.g. the memo:Re P0/P1/P2 rows from 2026-05-16 that sat ~5 days on the native side before the 2026-05-21 D266 batch caught up) -- `~/src/graphrefly-rs/docs/migration-status.md` (every time, no exceptions) -- `~/src/graphrefly-rs/docs/porting-deferred.md` (every time, no exceptions) -- `~/src/graphrefly-rs/CLAUDE.md` (Rust-specific invariants) -- Any files the user referenced in $ARGUMENTS -- Existing Rust source for the area (`~/src/graphrefly-rs/crates//src/`) -- Existing Rust tests for the area (`~/src/graphrefly-rs/crates//tests/`) -- `~/src/graphrefly-ts/archive/docs/SESSION-rust-port-architecture.md` if entering a new milestone (M1→M2 transition, etc.) - -### Rust-specific invariants (validate proposed changes against these) - -These come from `~/src/graphrefly-rs/CLAUDE.md` and are non-negotiable: - -1. **No `unsafe`. Anywhere. Enforced by `#![forbid(unsafe_code)]` at every crate root.** If a feature seems to need unsafe, find a safe abstraction (parking_lot, dashmap, imbl, redb, napi-rs / pyo3 wrappers). Escalate before allowing the lint. -2. **Compiler-enforced thread safety.** `Send` + `Sync` discipline applies to every public type. No `Rc` / `RefCell` in shared state — use `Arc` + `Mutex` / `parking_lot::ReentrantMutex`. -3. **Per-subgraph `parking_lot::ReentrantMutex`** (planned; mirrors graphrefly-py per-subgraph RLock parity goal). -4. **No async runtime in Core.** Core dispatcher is sync. `tokio` only enters in `graphrefly-storage` and bindings; never in `graphrefly-core`. -5. **No `unwrap()` / `expect()` on user-facing paths.** Domain errors via `thiserror`-derived enums. `unwrap` only in tests, build scripts, or impossible-by-construction paths (with comment). -6. **`#[must_use]` on every public fn that returns a value.** -7. **`clippy::pedantic` + `rust_2018_idioms` warn-by-default.** Allow on a per-need basis with a comment, never silently. -8. **Public types live behind newtype wrappers** (`NodeId(u64)`, `HandleId(u64)`, etc.). Don't expose raw integers — they collide structurally. -9. **Snapshot serialization uses `serde_ipld_dagcbor`** for content-addressed paths, `ciborium` for non-content-addressed snapshots. Never mix codec choice with content-addressing semantics. - -### Cross-language invariants (also apply to Rust port) - -- **Handle-protocol cleaving plane.** Core operates on opaque `HandleId` integers. User values `T` live in the binding-side registry; they never enter Core types. The `BindingBoundary` trait is the only mandatory FFI crossing per fn-fire. -- **No polling.** Use reactive timer sources, not `std::thread::sleep` loops or `tokio::interval` busy-checks against state. -- **No imperative triggers in public API.** All coordination via message flow. Imperative methods only on the L2.35 controller-with-audit primitives. -- **First-run gate** (R2.5.3) — compute node does NOT fire fn until every dep has delivered at least one real handle. -- **Equals-substitution under `equals: 'identity'` is zero-FFI** — `HandleId` u64 compare in pure Rust. Custom equals crosses the binding boundary; opt-in only. -- **DIRTY before DATA/RESOLVED** (R1.3.1.b two-phase push) in the same wave. -- **Tier ordering** (R1.3.7.b) — Tier 0 START, Tier 1 DIRTY, Tier 2 PAUSE/RESUME, Tier 3 DATA/RESOLVED, Tier 4 INVALIDATE, Tier 5 COMPLETE/ERROR, Tier 6 TEARDOWN. Tier 3+4 buffer under PAUSE; others bypass. -- **§10 simplifications** (from `archive/docs/SESSION-rust-port-architecture.md` Part 10) — apply where they fit the slice; do NOT blindly transliterate TS patterns. Defer perf-tier §10 items (10.3 / 10.4 / 10.5 / 10.6 / 10.13) until bench evidence justifies; record deferrals in `porting-deferred.md`. - -### When to compare against TS for parity vs spec - -- **Behavior parity:** the Rust port must satisfy the same invariants as the canonical spec. The TS production code is a **reference for behavior**, not a structural template. When TS code disagrees with the canonical spec, **the canonical spec wins** (see §11 Implementation Deltas — explicit list of TS-vs-canonical drift). -- **Test parity:** when porting a feature with TS tests, mirror the test scenarios in Rust. Each test should reference the canonical spec rule it covers (e.g., a comment or test name like `r2_6_4_teardown_auto_precedes_complete`). -- **Bench parity:** if the slice claims a perf win, validate via criterion bench. Pre-existing `dispatcher.rs` bench shapes are the canonical comparison harness. - -Do NOT start implementing yet. - ---- - -## Phase 2: Architecture Discussion - -### Full mode — HALT - -**HALT and report to the user before implementing.** Present: - -1. **Current state confirmation** — what's already in `graphrefly-rs` for this area (cite migration-status.md milestone status; cite specific files / line ranges; verify `cargo nextest run -p ` is clean before changes). -2. **Slice scope** — what the slice will and will NOT include. Slices should be: - - Coherent (single feature or audit-fix bucket) - - Reasonably small (~500–2000 lines including tests for a typical M1-style slice) - - Tied to a milestone via the `migration-status.md` table - - Aligned with §10 simplifications where applicable (call out which ones apply, which defer) -3. **Architecture choices** — for each new public API: signature, error variants, lock discipline, refcount discipline, handle-protocol boundary semantics. Cite canonical spec rules (`R`) for behavior decisions. -4. **Open questions** — surface any spec ↔ canonical-spec ↔ TS-drift conflicts BEFORE coding. Per the user-feedback memory: "no autonomous decisions — surface spec↔code conflicts instead of silently picking." -5. **Acceptance bar** — what needs to be green before the slice closes: - - All existing tests still pass - - New tests cover the canonical-spec rules touched - - `cargo clippy -p --all-targets` clean - - `cargo fmt --check` clean - - `#![forbid(unsafe_code)]` preserved - - `migration-status.md` updated - - New limitations / divergences added to `porting-deferred.md` - -Prioritize (in order): -1. **Spec alignment** — matches `docs/implementation-plan-13.6-canonical-spec.md` (canonical post-13.6.A). Where canonical disagrees with current TS impl, follow canonical. -2. **Refcount + lock discipline** — Rust impl must NOT introduce refcount leaks or lock-ordering bugs. The §10.2 PauseState pattern (retain on buffer-push, release on drain/overflow) is the canonical example. -3. **Test coverage** — every public API surface gets a test. Edge cases (terminal interactions, pause cross-cuts, set_deps validation) get explicit tests. -4. **Consistency** — patterns elsewhere in `graphrefly-core` (RAII Subscription via `Weak>`, `parking_lot::Mutex`, single state lock). -5. **Simplicity** — don't pre-optimize. v1 single-mutex is fine; perf-tier §10 simplifications wait for bench evidence. - -Do NOT consider backward compatibility at this early stage (pre-1.0). - -**Cross-repo decision log:** If Phase 1–2 surface an architectural or product-level question (canonical-spec ambiguity, parity divergence, refcount discipline gap), record it under "Active work items" in `docs/optimizations.md` (the graphrefly-ts source of truth for cross-language decisions). Rust-specific deferrals go in `~/src/graphrefly-rs/docs/porting-deferred.md`. Mark cross-references both ways. - -**Decision logging:** For each question you ask during HALT, after the user answers, append the decision to `docs/rust-port-decisions.md` using the format: - -```markdown -### DXXX — [short title] -- **Date:** YYYY-MM-DD -- **Context:** [what prompted the question] -- **Options:** A) … B) … C) … -- **Decision:** [what user chose] -- **Rationale:** [why] -- **Affects:** [which modules/milestones] -``` - -**Wait for user approval before proceeding.** - -### Light mode — Skip unless escalation needed - -Proceed directly to Phase 3 **unless** Phase 1 reveals any of these: -- Changes to **message protocol** (new tier, new variant, payload semantics) -- Changes to **`BindingBoundary` trait** (new method, signature change) -- Changes to wave engine, batch coalescing, or dispatch order -- New public types in `graphrefly-core` (especially RAII handles, error enums) -- Multiple viable approaches with non-obvious trade-offs -- Drift between canonical spec and current `graphrefly-rs` impl that needs an explicit reconciliation call -- Touching anything flagged in `porting-deferred.md` as a deferred concern - -If any apply, escalate: HALT and present findings as in full mode. - ---- - -## Phase 3: Implementation & Self-Test - -After user approves (full mode) or after Phase 1 (light mode, no escalation): - -### 3a. Implement - -1. Treat `docs/implementation-plan-13.6-canonical-spec.md` as non-negotiable for behavior. If existing Rust code drifts from canonical, align toward canonical as part of the change. -2. Cross-reference rule IDs in code comments where the design is non-obvious (e.g., `// R2.6.4 / Lock 6.F: TEARDOWN auto-precedes COMPLETE`). -3. Apply §10 simplifications where they fit the slice; defer perf-tier ones with a note in `porting-deferred.md`. -4. Maintain the handle-protocol cleaving plane: Core sees `HandleId`, never `T`. If a temptation arises to leak `T` into Core (e.g., for debugging), use a `BindingBoundary::deref_for_debug` shape instead. -5. Refcount discipline: - - Every `retain_handle` paired with a `release_handle`. - - When buffering a handle (PauseState, terminal slot, dep_terminals slot), retain on store, release on remove. - - Cross the boundary OUTSIDE the state lock when feasible (mirrors `Core::resume` Phase 2 pattern), to avoid binding-vs-Core lock ordering issues. - -### 3b. Tests - -1. Put tests in the most specific existing file under `~/src/graphrefly-rs/crates//tests/`. Common patterns: - - One file per feature: `pause.rs`, `invalidate.rs`, `terminal.rs`, etc. - - Use the shared `tests/common/mod.rs` `TestRuntime` + `TestBinding` + `Recorder` infrastructure. - - Use `RecordedEvent::*` for high-level message assertions (resolves handles to values automatically). -2. Cover the edge cases the canonical spec calls out — e.g., R1.4 idempotency-within-wave for INVALIDATE; R2.6.4 idempotency on duplicate TEARDOWN. -3. For refcount-touching changes, use `TestBinding::refcount_of(handle)` to verify retain/release pairs balance (don't rely on `live_handles()` alone — handles stay alive when ANY share remains). -4. Test names should reference the canonical rule when covering a specific invariant (e.g., `dynamic_rewire_refires_fn_on_new_deps` covers a Phase 13.8 audit fix; `r1_4_invalidate_idempotent_within_wave` covers a spec rule). - -### 3c. Self-check - -Run from `~/src/graphrefly-rs`: - -```bash -cargo nextest run --profile ci -p graphrefly-core # core tests, incl. cascade_depth guards -cargo nextest run --profile ci # default-members, full suite (== `cargo tc`) -cargo clippy -p graphrefly-core --all-targets -cargo fmt --check # or `cargo fmt && cargo fmt --check` -``` - -When the slice touches bindings: -```bash -cd crates/graphrefly-bindings-js && pnpm build # napi-rs build -cd crates/graphrefly-bindings-py && maturin develop # pyo3 build -cd crates/graphrefly-bindings-wasm && wasm-pack build -``` - -Fix any failures. **Do NOT use `--workspace`** for `cargo build` / `cargo nextest run` unless you have all binding toolchains installed — the workspace excludes bindings from default-members for this reason. (Self-check uses `--profile ci` to run the full suite incl. the `cascade_depth` stack-safety guards before closing a slice; the quarantined default profile is for the inner loop only.) - -> **Run the self-check via the sanctioned long-runner, synchronously.** Use `mise run gate` / `mise run gate:core` (or `mise run run-logged -- `) and **wait foreground for the `<<>>` sentinel** — never background cargo and monitor a non-guaranteed string. **Subagent hygiene:** if this skill runs in a spawned subagent, it MUST run verification synchronously or tear down any backgrounded command (kill by process group) **before returning** — a live background process leaks as a stale parent-session "running" entry indistinguishable from a real hang. See `~/src/graphrefly-ts/docs/test-guidance.md` § "Running long commands reliably / diagnosing a stuck run" and memories `feedback_long_command_observation.md` / `feedback_subagent_bg_hygiene.md`. - -### 3d. Widen the parity-test surface (when slice closes a milestone or adds public API) - -If the slice closes (or partially fills) a milestone row in the `packages/parity-tests/README.md` schedule table, add cross-impl scenarios under `~/src/graphrefly-ts/packages/parity-tests/scenarios//`: - -1. Pick the layer subfolder (`scenarios/core/` for M1 dispatcher, `scenarios/graph/` for M2 Slice E, `scenarios/operators/` for M3, etc. — create the folder if needed). -2. Write the scenario as `describe.each(impls)(" parity — $name", (impl) => { test(...); })`. Reference symbols only via `impl.`, not direct imports — that's what makes the scenario impl-agnostic. -3. If the scenario references new symbols not yet in `packages/parity-tests/impls/types.ts` `Impl`, widen the interface (and provide the field on `pureTsImpl` in `impls/pure-ts.ts`). Until `@graphrefly/native` publishes, `rustImpl` is `null` and scenarios only run against `pureTsImpl` — but the parameterization stays in place so activation only requires a one-line `rust.ts` flip. -4. Test: `pnpm --filter @graphrefly/parity-tests test`. Scenario must pass against `pureTsImpl`. When `rustImpl` activates later, mismatches fail loud. - -**Skip this step** if the slice is an internal refactor that doesn't change public surface (e.g., a §10 perf simplification under an unchanged API). The parity-tests layer is for surface-visible behavior, not internals. - -### 3e. Document the slice - -Update both Rust-side operational docs as the work lands: - -1. **`~/src/graphrefly-rs/docs/migration-status.md`** — the canonical milestone tracker: - - When a sub-bucket lands: mark it ✅ in the M-table or the entry checklist. - - When a milestone closes: update the M-table row to ✅, add a `## M — closed YYYY-MM-DD` section documenting what landed, what was deferred, what was carried forward. - - Update the test count (per-file breakdown helps future readers). - - Cross-reference any cargo-tagged release: `git tag -a vM.0.0 -m "M complete"` + bump `[workspace.package].version`. - -2. **`~/src/graphrefly-rs/docs/porting-deferred.md`** — the running registry of deferred concerns: - - When the slice surfaces a new perf concern that you DON'T fix: add a section with what / why-deferred / source. - - When the slice surfaces a v1 dispatcher limitation: add to "v1 dispatcher limitations". - - When the slice acknowledges a TS-spec divergence: add to "Spec divergences acknowledged in v1". - - When the slice CLOSES a previously-deferred concern: move that entry's content to "Audit fixes landed in Slice X" (or the milestone's closing section in `migration-status.md`) and remove from the active deferred list. - -3. **TS-side `docs/implementation-plan.md`** — only when a TS-side phase entry is affected (e.g., a Phase 13.7 / 13.8 sub-item closing). The Rust port does NOT add Phase 11–16 sub-bullets; the migration-status.md is the canonical Rust tracker. - -4. **TS-side `docs/optimizations.md`** — only when the slice surfaces a cross-language design question that needs to be tracked alongside TS / PY work. Rust-only deferrals go in `porting-deferred.md`, not `optimizations.md`. - -### 3f. Closing the slice - -When done, produce these deliverables: - -**A. Behavioral trace table** — for each new/changed module, show a plain-English table: - -``` -Module: [name] (milestone) -Scenario: [description of the most representative scenario] - -Step | Event | Internal state change | Observable output -1 | ... | ... | ... -``` - -The user verifies this against the spec without reading Rust. If the trace is correct AND parity tests pass, the impl is correct. - -**B. Simplification delta** — table showing what the Rust version does differently from TS: - -| TS pattern | Rust replacement | Why simpler / Why different | -|---|---|---| - -Flag any entry where Rust is MORE complex than TS — that's a potential over-engineering signal. - -**C. Deferred item stubs** — for each new deferred item, confirm a `#[ignore]` test exists in the Rust source: - -```rust -#[test] -#[ignore = "deferred: ()"] -fn () { /* impl when feature lands */ } -``` - -**D. Standard closing checklist:** - -1. List files changed and new public types / methods added. -2. Cite the migration-status.md row this slice closes (or moves toward closing). -3. Cite the canonical-spec rules covered. -4. Cite any new entries in porting-deferred.md. -5. Suggest running `/qa` for adversarial review and final checks. - -If implementation **closes a milestone** in `migration-status.md`: -1. Move the milestone's row from "🚧 in progress" to "✅ landed". -2. Add the closing section per the existing template (see M1 dispatcher / M1 parity sections for the canonical format). -3. Sweep `porting-deferred.md` for items resolved by this milestone — move them to the closing section. - -If implementation **closes a Rust-side milestone-pre-condition** in `docs/implementation-plan.md` (Phase 13.7 / 13.8 / similar): mark ✅ in the matching TS-side phase entry per `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/implementation-plan.md`". - ---- - -## Quick reference: typical slice flow - -1. Read canonical spec section + flowchart batch covering the feature -2. Read migration-status.md to know the milestone context + what's landed -3. Read porting-deferred.md to know what NOT to re-introduce -4. (Full mode) Halt with architecture proposal citing R rules -5. (User approval) Implement + tests + clippy + fmt -6. **If the slice closes a milestone or adds public API:** widen `~/src/graphrefly-ts/packages/parity-tests/scenarios//` with new `describe.each(impls)` scenarios; verify `pnpm --filter @graphrefly/parity-tests test` green -7. Update migration-status.md + porting-deferred.md -8. Suggest `/qa` diff --git a/.claude/skills/rust-review/SKILL.md b/.claude/skills/rust-review/SKILL.md deleted file mode 100644 index 97caa479..00000000 --- a/.claude/skills/rust-review/SKILL.md +++ /dev/null @@ -1,236 +0,0 @@ ---- -name: rust-review -description: "Post-implementation quality review for Rust port slices. Runs the audit-data extractor, then appends one structured row to reviews.jsonl + new rows to findings.jsonl + (if applicable) edits flowcharts.md. Output is data, not prose — every behavioral trace, simplification delta, and finding lands in the audit dashboard. Use after /porting-to-rs completes a slice, or standalone when you want to verify a Rust module's correctness without reading Rust." -argument-hint: "[module or slice name, e.g. 'batch coalescing' or 'M1 Slice C']" ---- - -You are executing the **rust-review** workflow. Output is **structured data**, never markdown prose. Every analytical artifact lands in JSONL files under `~/src/graphrefly-rs/docs/audit/data/`, which the audit dashboard at `http://localhost:8769/audit/site/` renders. - -Target: $ARGUMENTS - ---- - -## Phase 0: Refresh derived data - -Re-run the extractor first so you start the review against current source state: - -```bash -cd ~/src/graphrefly-rs && mise run audit-extract -# or directly: python3 docs/audit/extract.py -``` - -This regenerates the **derived** files (overwritten on every run): -- `items.jsonl` — every public/crate-private item with `loc`, `attrs`, `rules_cited` -- `rules.jsonl` — canonical spec rules (262 today) -- `tests.jsonl` — every `#[test]` fn with `covers_rules` extracted from doc-comment + body -- `topology.jsonl` — crate-to-crate `use` and `ref` edges -- `locks.jsonl` — lock acquisition sites (`Mutex::lock`, `RwLock::read|write`, etc.) -- `flowcharts.jsonl` — Mermaid diagrams from `docs/flowcharts.md` with metadata + source - -It does **not** touch `findings.jsonl` and `reviews.jsonl` — those are append-only, author-edited audit artifacts. - ---- - -## Phase 1: Load context - -Read in parallel: - -- `~/src/graphrefly-rs/docs/migration-status.md` — what's claimed as landed -- `~/src/graphrefly-rs/docs/porting-deferred.md` — known gaps -- `~/src/graphrefly-rs/docs/flowcharts.md` — Rust-port-specific shape diagrams (canonical Mermaid source) -- `~/src/graphrefly-ts/docs/implementation-plan-13.6-canonical-spec.md` — sections relevant to $ARGUMENTS -- `~/src/graphrefly-ts/docs/implementation-plan-13.6-flowcharts.md` — TS spec semantics -- `~/src/graphrefly-rs/docs/rust-port-decisions.md` — locked decisions -- The Rust source files for the module (`~/src/graphrefly-rs/crates//src/`) -- The Rust test files (`~/src/graphrefly-rs/crates//tests/`) -- `~/src/graphrefly-rs/docs/audit/data/findings.jsonl` — open findings on $ARGUMENTS to avoid duplicating -- `~/src/graphrefly-rs/docs/audit/data/reviews.jsonl` — past reviews of nearby slices - -Open the dashboard during the review: -``` -mise run audit-serve # background server -http://localhost:8769/audit/site/ # the live data view -``` - ---- - -## Phase 2: Behavioral traces - -Author 2–5 traces covering: the happy path, the most complex edge case (diamond, pause interaction, terminal cross-cut), and any scenario the parity tests cover. Each trace is a structured object (not markdown). Schema: - -```json -{ - "id": "T1", - "title": "Pause-overflow ERROR synthesis", - "rules": ["R1.3.8.c", "Lock 6.A"], - "diagrams": ["11.1"], - "steps": [ - {"step": 1, "event": "set_pause_buffer_cap(node, Some(2))", "internal": "NodeRecord.pause_buffer_cap = 2", "output": "—"}, - {"step": 2, "event": "...", "internal": "...", "output": "..."} - ], - "commentary": "Pre-Slice-F this was a silent drop. Now structured ERROR with {nodeId, droppedCount, configuredMax, lockHeldDurationMs}." -} -``` - -Build the trace objects in your head, hold them — they get embedded in the `reviews.jsonl` row in Phase 6. - ---- - -## Phase 3: Simplification delta - -For each public-facing change in the slice, capture a delta row: - -```json -{ - "n": 1, - "ts_pattern": "TS pause overflow: silent drop or implementation-defined", - "rust_replacement": "Synthesized Error with structured diagnostic", - "simpler": "same", // "same" | "yes" | "no" | "rust-harder" - "notes": "Slice F A3 — closes documented divergence", - "diagram": "11.1" // optional flowchart id -} -``` - -Flag `simpler: "no"` or `"rust-harder"` rows as potential over-engineering. If a row's complexity is justified (type safety, thread safety, etc.) put the justification in `notes`. If unjustified, plan to also emit a `kind:"opp"` finding in Phase 5. - ---- - -## Phase 4: Deferred gap audit - -For each `#[ignore]` test in the module, check it has a matching entry in `porting-deferred.md`. For each deferred-but-not-ignored item, plan a `kind:"limit"` finding in Phase 5. For each item that's silently depended on for correctness, plan a `kind:"bug"` finding. - ---- - -## Phase 5: Findings - -Each issue surfaced becomes one row in `findings.jsonl`. Re-use a draft authored via the dashboard's "+ New finding" drawer (fastest path — IDs auto-increment from the highest existing `F`), or hand-author. Schema: - -```json -{ - "id": "F011", - "kind": "bug", // bug | limit | opp | note | complete-gap - "severity": "major", // critical | major | minor - "title": "Diamond resolution overflows at fan-in >32", - "where": "crates/graphrefly-core/src/dispatcher.rs", - "where_line": 142, - "rule": "R5.8", // optional - "slice": "M3 Slice F audit /qa D4", // optional - "evidence": "Reproducer: tests/wide_fanin.rs::T_diamond_w33. Bitmask is u32 → silently overflows.", - "recommendation": "Lift bitmask to u128 for fan-in ≤128, fall back to Vec chunks above.", - "status": "open", // open | closed | superseded | draft - "opened_at": "2026-05-09", - "closed_at": null, - "supersedes": null, - "source": "rust-review (slice F audit)" -} -``` - -**Authoring tips**: -- Use the dashboard's authoring drawer for any finding tied to a specific file — the `where` field gets autocomplete. -- One finding per issue. Don't bundle. -- For "Rust simpler than TS" wins worth surfacing, use `kind:"opp"`. -- For deferred-by-design items, use `kind:"limit"` and put the deferral reason in `evidence`. -- For missing parity scenarios, use `kind:"complete-gap"` and cite the `rule` it would test. - ---- - -## Phase 6: Append the review row - -When traces, deltas, and findings are all authored, build **one** `reviews.jsonl` row that ties them together and append it (no rewrites — the file is append-only): - -```json -{ - "id": "rev-2026-05-09-slice-q", - "review_date": "2026-05-09", - "slice": "M3 Slice Q", - "scope": "...", - "sha": "", - "tests_before": 438, - "tests_after": 471, - "premise": "", - "traces": [ ... Phase 2 trace objects ... ], - "deltas": [ ... Phase 3 delta rows ... ], - "findings_opened": ["F011", "F012", "F013"], - "assessment": { - "spec_fidelity": "very high", - "over_engineering": "low", - "correctness_holes": "none", - "halt": "no", - "summary": "" - }, - "source": "rust-review" -} -``` - -Use absolute dates (`2026-05-09`), not "today". `id` follows `rev--`. Append with: - -```bash -echo '' >> ~/src/graphrefly-rs/docs/audit/data/reviews.jsonl -``` - -(One line, no trailing comma, valid JSON.) - ---- - -## Phase 7: Maintain `flowcharts.md` - -`~/src/graphrefly-rs/docs/flowcharts.md` is still the **canonical Mermaid source**. The extractor reads from it; the dashboard renders from `flowcharts.jsonl`. - -For **each new public method, state machine, or distinctive pattern** you traced in Phase 2 that isn't already diagrammed: - -1. Add a Mermaid block under the appropriate batch heading (`## Batch N — title`) with a `### x.y title` heading. The extractor will pick it up next time you run `mise run audit-extract`. -2. Cite the spec rule (`R`) in the title or prose so `rules_cited` populates correctly. -3. Use existing conventions: 🟨 YELLOW for v1 limitations, 🟦 BLUE for Rust-specific simplifications, solid arrows for control flow, dashed for data flow. -4. Update the cross-reference tables at the end of the file (slice→diagram, spec rule→diagram, deferred→diagram). - -**For diagrams that became stale**: edit in place, add a short "updated in Slice X" note in the prose, no new diagram needed. - -**Mermaid syntax pitfalls** (verified to fail in mermaid v10): -- `;` in sequenceDiagram message text (use `,` or `—`) -- `:` inside stateDiagram-v2 state descriptions when followed by `{...}` (use `note right of ` blocks) -- Generics with `` inside class/state diagrams — use `~T~` -- Stray `activate`/`deactivate` pairs across `alt`/`else` branches — close every branch -- Bracket labels `["…"]` containing parens or `::` — replace with `[…]` plain text - -After Mermaid edits, **re-run the extractor** so `flowcharts.jsonl` updates: - -```bash -mise run audit-extract -``` - ---- - -## Phase 8: Verify in the dashboard - -1. `mise run audit-serve` (or `preview_start` with the `rust-audit` profile in `.claude/launch.json`). -2. Open `http://localhost:8769/audit/site/`. -3. Spot-check each tab: - - **Reviews** → your new row sits at the top (newest first), expand it, confirm traces + deltas + findings render correctly. - - **Findings** → new rows appear with the right `where` clickable to Repo Map. - - **Flowcharts** → any new diagram appears in its batch group; click to render Mermaid. - - **Spec ⇄ Impl** → if the slice changed citations, the matrix shows updated counts; rules touched by new findings have the bug indicator; flowchart-citing rules show the 📊 chip. - - **Repo Map** → drill into any file you logged a finding on; verify the sidecar shows the new finding. -4. Note in the conversation any tabs that didn't update as expected. - -> **Subagent / background hygiene.** `mise run audit-serve` is a long-lived background server; if you reproduce a finding with a Rust command, run it through `mise run run-logged -- ` (or `mise run gate:core`) and wait foreground for the `<<>>` sentinel — never monitor a non-guaranteed string. If this skill runs in a spawned subagent, it MUST stop the audit server / tear down any backgrounded command (kill by process group) **before returning** — a live background process leaks as a stale parent-session "running" entry indistinguishable from a real hang. See `~/src/graphrefly-ts/docs/test-guidance.md` § "Running long commands reliably / diagnosing a stuck run" and memory `feedback_subagent_bg_hygiene.md`. - ---- - -## When to escalate - -If during the review you find: -- A behavioral trace that **contradicts** the canonical spec → HALT, surface the contradiction, write a `kind:"bug" severity:"critical"` finding, do NOT append the review row until the user resolves. -- A delta row where Rust adds machinery that's NOT justified by a Rust-specific constraint → write a `kind:"opp"` finding with `recommendation:` listing the simpler shape. -- A deferred item the current code silently depends on for correctness → flag as `kind:"bug"` (not `limit`) — that's a hidden hole. -- A flowchart in `flowcharts.md` that contradicts current source (drift) → fix the diagram in Phase 7 and call out the drift in the review's `premise`. -- A Mermaid render failure you can't fix in two attempts → leave the diagram in but cite it in the review's `assessment.summary` so it gets followup attention. - ---- - -## What changed from the old workflow - -The pre-2026-05-09 SKILL wrote a markdown report (`reports-NNN-.md`) under `docs/review/`. Those files are **frozen historical artifacts** — leave them. New reviews go to `docs/audit/data/reviews.jsonl` only. - -The Phase 6.5 directive vocabulary (`::: trace`, `::: finding`, …) used by the legacy report renderer is no longer used. The structured shape lives in JSONL fields directly. - -The legacy site renderer at `docs/review/site/` was **removed 2026-05-24** (moved to `~/src/graphrefly-rs/TRASH/review-site-removed-2026-05-24/`) to eliminate dual-dashboard confusion. The 6 historical reports at `docs/review/reports-*.md` are preserved as flat-file markdown — every `findings.jsonl` row's `source` field still cites them by filename. Read them in any editor; do not edit them; do not resurrect the renderer. diff --git a/.claude/skills/spec-amend/SKILL.md b/.claude/skills/spec-amend/SKILL.md new file mode 100644 index 00000000..abed49e4 --- /dev/null +++ b/.claude/skills/spec-amend/SKILL.md @@ -0,0 +1,34 @@ +--- +name: spec-amend +description: "Spec-first protocol amendment flow for the clean-slate GraphReFly redesign. Use BEFORE changing any wave-protocol behavior (tiers, wave semantics, diamond/equals/SENTINEL, batch, push-on-subscribe, ctx.up/down contract). Enforces F-NO-IMPL-DEFINED: amend spec/rules.jsonl + formal/*.tla + spec/conformance.jsonl FIRST, then implement in each language package. Triggers: 'amend the spec', 'change the protocol', 'add a tier', 'spec change', 'new wave rule', 'this changes protocol behavior'. NOT for sugar/operator/inspection changes — those are per-language, never touch spec." +argument-hint: "[short description of the protocol behavior to change]" +--- + +You are executing **spec-amend** for the clean-slate GraphReFly redesign. + +**Authority repo:** `~/src/graphrefly` (clean-slate branch) holds the language-neutral spec. +Per-language packages (`graphrefly-{ts,rust,py}`) implement it; they NEVER define protocol behavior. + +## Iron rule (F-NO-IMPL-DEFINED, decision D14/D19) + +Protocol behavior is **spec-first**. No "implementation defines what happens." Order is fixed: + +1. **Amend the spec data** (before any code): + - `~/src/graphrefly/spec/rules.jsonl` — add/edit the normative rule (`{id, area, tier?, statement, rationale, status, since:"D#", covers_by:[]}`). Mark `status:"draft"` until conformance + code land, then `"active"`. + - `~/src/graphrefly/formal/*.tla` (+ MC config) — model the behavior; add the invariant; run TLC. (formalization γ, D14.) + - `~/src/graphrefly/spec/conformance.jsonl` — add the behavioral scenario(s) that pin the new rule (`covers:[rule-id]`, `runtimes:{ts:"todo",rust:"todo",py:"todo"}`, `status:"required"`). +2. **Record the decision** if this is a new architectural lock: append a `D#` to `~/src/graphrefly/decisions/decisions.jsonl` (or reference the existing one in `since`). +3. **Run the consistency gate:** `node ~/src/graphrefly/dashboard/build.mjs --check` (no broken links/orphans). +4. **THEN implement** in each language package to make the conformance scenarios pass; flip `runtimes.` → `"pass"` as each lands. Use `/dev-dispatch` per package. + +## Closed-set guardrails (do not bypass) + +- **9 tiers are a closed set** (D9). Adding a tier is a constitutional change — requires explicit user lock + TLA+ re-model, not a casual amend. +- **onMessage/onSubscribe are substrate-fixed** (D19) — they are NOT user-replaceable hooks; "amend" means changing the spec'd behavior, not adding a config knob. +- **equals fires only single-DATA-wave** (D15); **ctx.up is control-tier only** (R-ctx-up); **restore ≠ fresh-lifecycle wipe** (R-restore). Re-read these rules before touching adjacent behavior. + +## Output + +A spec-amendment plan: which rule(s) change, the TLA+ invariant delta, the conformance scenario(s) added, the D# (new or referenced), and the per-language implementation order. HALT for user approval before writing TLA+/code if the change touches a closed-set guardrail. + +After the spec data lands and `--check` is clean, hand off to `/dev-dispatch` per language package and `/conformance` to drive the scenarios green. diff --git a/CLAUDE.md b/CLAUDE.md index 0e920099..006827b0 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,214 +1,82 @@ -# graphrefly — unified agent context - -**GraphReFly** — reactive graph protocol for human + LLM co-operation. This repo (`graphrefly-ts`) is the **single source of truth** for operational docs, skills, roadmap, and optimization records across both the TypeScript and Python implementations. - -## Repos - -| Repo | Path | Role | -|------|------|------| -| **graphrefly-ts** | this repo | TypeScript implementation + **all operational docs** | -| **graphrefly-py** | `~/src/graphrefly-py` | Python implementation (must stay in parity) | -| **graphrefly** (spec) | `~/src/graphrefly` | `GRAPHREFLY-SPEC.md`, `COMPOSITION-GUIDE.md` | -| **callbag-recharge** | `~/src/callbag-recharge` | TS predecessor (patterns/tests, NOT spec authority) | -| **callbag-recharge-py** | `~/src/callbag-recharge-py` | PY predecessor (concurrency patterns, subgraph locks) | - -## Canonical references (read these) - -| Doc | Role | -|-----|------| -| `~/src/graphrefly/GRAPHREFLY-SPEC.md` | **Behavior spec** — messages, `node`, `Graph`, invariants | -| `~/src/graphrefly/COMPOSITION-GUIDE.md` | **Composition guide** — insights, patterns, recipes for Phase 4+ factory authors. **Read before building factories that compose primitives.** Covers: lazy activation, subscription ordering, null guards, feedback cycles, promptNode SENTINEL, wiring order. | -| `docs/implementation-plan.md` | **CANONICAL pre-1.0 sequencer** — Phases 11–16 + Parked + Open design sessions. Tier 1–10 historical record + the active Phase 11–16 plan locked 2026-04-30 (cleanup → consolidation → multi-agent → changesets/diff → roadmap residuals → eval → launch). Read this FIRST when picking up "what's next." Phase 13 covers multi-agent + intervention substrate (sources: `archive/docs/SESSION-multi-agent-gap-analysis.md` + `SESSION-human-llm-intervention-primitives.md`). | -| `docs/optimizations.md` | **Active backlog (line-item state)** — open work items, anti-patterns, deferred follow-ups, proposed improvements. Item-level provenance for entries that the implementation-plan.md phases reference. Add new items here. | -| `archive/optimizations/` | **Optimizations archive** — built-in optimizations, resolved design decisions, cross-language parity notes. Check before introducing new optimizations or debugging perf issues. **Backlog/proposed items belong in `docs/optimizations.md`, not here.** | -| `docs/cross-track-ledger.md` | **Cross-track coordination ledger** — the single place to log any change that widens the `Impl` contract (`packages/parity-tests/impls/types.ts`) or otherwise couples the presentation track (`@graphrefly/graphrefly`) ↔ Rust-port track (`@graphrefly/native`). Add a row BEFORE landing such a change (N1 → graphrefly-rs migration-status item-8 is the handoff pattern). Not for general optimization items — those stay in `docs/optimizations.md`. | -| `docs/roadmap.md` | **Vision / wave context** (no longer the active sequencer per 2026-04-30 migration — see `implementation-plan.md`). Useful for the strategic frame: Wave 0/1/2/3 announcement structure, harness engineering positioning, eval-story narrative. New items go to `implementation-plan.md`, not here. | -| `docs/docs-guidance.md` | How to document APIs and long-form docs (covers both TS and PY) | -| `docs/test-guidance.md` | How to write and organize tests (covers both TS and PY) | -| `archive/docs/SESSION-graphrefly-spec-design.md` | Design history and migration from callbag-recharge | -| `archive/docs/SESSION-reactive-collaboration-harness.md` | **Active** — 7-stage reactive collaboration loop (INTAKE→TRIAGE→QUEUE→GATE→EXECUTE→VERIFY→REFLECT), gate port from callbag-recharge, `promptNode` factory, `valve` rename, strategy model (`rootCause × intervention → successRate`), `harnessLoop()` factory. Source of truth for §9.0. | -| `archive/docs/SESSION-DS-14.5-A-narrative-reframe.md` | **Active (canonical post-2026-05-04)** — spec-as-projection reframe, multi-agent subgraph ownership protocol (L0–L3 staircase), catalog reframed as user-host concern, Wave 2 narrative shift away from "harness builder". L1–L8 + Q1–Q10 locks. **Read this before editing README / Wave 2 launch copy.** | -| `archive/docs/SESSION-DS-14-changesets-design.md` | **Active (locked 2026-05-05)** — universal `BaseChange` envelope, `mutations` companion bundles, `mutate(act, opts)` factory, lifecycle-aware diff restore. Substrate for op-log changesets / worker-bridge wire B / `lens.flow` delta / `reactiveLog.scan` / `restoreSnapshot mode "diff"`. Source of truth for Phase 14 implementation. | -| `archive/docs/SESSION-harness-engineering-strategy.md` | **SUPERSEDED 2026-05-04 by DS-14.5.A** for Wave 2 narrative framing. Original 8-requirement coverage analysis + harness engineering landscape preserved as historical context. New positioning lives in `SESSION-DS-14.5-A-narrative-reframe.md`. | -| `archive/docs/SESSION-marketing-promotion-strategy.md` | **Active** — positioning pillars (pain-point-first), wave-based announcement plan, pain-point reply marketing playbooks, xiaohongshu strategy, Future AGI competitive intel (§16), prompt optimization algorithm analysis (§17), blog content plan (§18). Source of truth for public-facing copy. **Wave 2 framing should rebase on DS-14.5.A.** | +# graphrefly-ts — agent context (TypeScript implementation) + +**GraphReFly** — reactive universal reduction layer (high fan-in/out → information reduction → push; +not LLM-limited, D1). This repo is the **TypeScript implementation** (`@graphrefly/ts`): a +self-contained package (substrate + sugar + operators), **no cross-language peer-deps** (D32). + +> **This file points, it does not host.** The language-neutral authority — protocol spec, +> decisions, design sessions, conformance, formal model — lives in `~/src/graphrefly` (branch +> `clean-slate`). When anything here disagrees with that repo, **that repo wins.** Do not +> duplicate its content back into this file. + +## Authority — where the truth lives (`~/src/graphrefly`) + +Read `~/src/graphrefly/CLAUDE.md` first — it is the single-source index for the design. + +| Concern | Source of truth | +|---|---| +| **Decisions (why)** — unified D# log | `~/src/graphrefly/decisions/decisions.jsonl` (read via `/decision-guard`) | +| **Design narrative** — full L0–L6 locks, F-* constraints, flags, spec-amendment list | `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) | +| **Protocol rules (宪法)** | `~/src/graphrefly/spec/rules.jsonl` (changed via `/spec-amend`) | +| **Conformance scenarios (parity)** | `~/src/graphrefly/spec/conformance.jsonl` (driven via `/conformance`) | +| **Formal model** | `~/src/graphrefly/formal/*.tla` (+ MC configs) | +| **Sequencer (what next) / backlog / anti-patterns** | `~/src/graphrefly/plan/{phases,backlog,antipatterns}.jsonl` | +| **Guides (composition / docs / test / contribute)** | `~/src/graphrefly/guide/guide.jsonl` | +| **Rendered view** (progress / structure / gaps / search) | `~/src/graphrefly/dashboard/` (`node dashboard/build.mjs`) | + +Sibling implementations (each self-contained, cross-language = wire bridge, not in-process): +`@graphrefly/rust` (`~/src/graphrefly-rs`), `@graphrefly/py` (`~/src/graphrefly-py`). + +## Clean-slate floor (cite, never violate — full text in DS-1 / `rules.jsonl`) + +- **Sacred (L0.7):** topology declarative/serializable/inspectable · wave protocol is a public spec · + wave protocol impl is **sync** · all fn go through the dispatcher. +- **8 verbs, closed set (D4):** `node` `graph` `batch` `state` + `producer` `derived` `effect` `mount`. + Operators are `node` sugar, not verbs — per-language, never in parity (D6). +- **`ctx.up` / `ctx.down(msgs)` (D8):** one `msgs` array = one wave; may mix tiers. `ctx.up` is + **control-tier only** (DIRTY/PAUSE/RESUME/INVALIDATE/TEARDOWN); DATA/RESOLVED/COMPLETE/ERROR are + down-only (R-ctx-up). Handle = pure data `(pool_id, handle_id)`, no methods (D7). +- **9 tiers + PAUSE/RESUME (D9, R-tier):** closed set; adding a tier is a constitutional change. +- **graph = single-thread causal/concurrency domain (D22):** parallelism via pool callback or + multi-graph + wire bridge; rewire intra-graph only. +- **parity = behavioral conformance (D24):** structural `Impl` + cross-track-ledger retired. +- **config dissolved (D26):** clock is graph-local (no global singleton); `messageTier` is a + compile-time const table; `onMessage`/`onSubscribe` are substrate-fixed, not user-replaceable (D19). +- **Forced (F-*):** F-SYNC-CORE (async lives only in pools / wire-bridge) · F-DISPATCH-ALL (no + inline-fn bypass) · F-NO-IMPL-DEFINED (spec-locked or explicitly undefined) · F-NO-WEDGE-CUT · + F-NO-LLM-ONLY · F-GRAPH-FIRST-API · F-PERF. + +Durable values (memory `feedback_*`): no backward compat (pre-1.0) · no imperative triggers · +single source of truth · **no autonomous decisions** (surface spec↔code conflicts, don't silently pick) · +no implement without explicit approval · verify premise before greenfield. + +## Workflow rules + +- **spec-first** (F-NO-IMPL-DEFINED): any protocol behavior change → amend `~/src/graphrefly` + `spec/rules.jsonl` + `formal/*.tla` + `spec/conformance.jsonl` **before** code (`/spec-amend`). +- **decision-first**: any architectural lock → a `D#` in `~/src/graphrefly/decisions/decisions.jsonl` + before code (`/design-review` → user approval → append). +- **consistency gate**: `node ~/src/graphrefly/dashboard/build.mjs --check` (non-zero on broken + links / orphans) after touching any spec/decision/plan jsonl. ## Commands -**TypeScript (this repo) — post-Phase-13.9.A cleave:** ```bash -pnpm test # pure-ts test suite + parity-tests -pnpm test:pure-ts # just packages/pure-ts (~2980 tests) -pnpm test:parity # just packages/parity-tests -pnpm run lint # biome check (workspace-wide) -pnpm run lint:fix # biome check --write -pnpm run build # pure-ts build → root shim build -pnpm run build:shim # only the shim (assumes pure-ts already built) -pnpm bench # pure-ts vitest bench +pnpm test # full TS test suite +pnpm run lint # biome + layer/typecheck gates +pnpm run lint:fix # biome check --write +pnpm run build # build the package +pnpm bench # vitest bench (informational, not a CI gate — L5-Q1) ``` -For watch-mode work inside the pure-ts package: `pnpm --filter @graphrefly/pure-ts test:watch`. +## Skills (clean-slate) -**Python (`~/src/graphrefly-py`):** -```bash -uv run pytest # tests -uv run ruff check src/ tests/ # lint -uv run ruff check --fix src/ tests/ # lint fix -uv run ruff format src/ tests/ # format -uv run mypy src/ # type check -``` - -Python workspace managed by mise. `mise trust && mise install` to set up uv. `uv sync` to install dependencies. Distribution name and import path: `graphrefly` (i.e. `pip install graphrefly` — the `graphrefly-py` name refers to the repo, not the published package). - -## Documentation workflow (critical) - -- `docs/docs-guidance.md` is the cross-language documentation standard. -- `website/src/content/docs/api/*.md` is generated output. Do not hand-edit. -- For API docs updates: - 1. Update source JSDoc/docstrings. - 2. Run docs generation in the respective repo (`pnpm --dir website docs:gen`). - 3. Validate with `pnpm --dir website docs:gen:check` and `pnpm --dir website sync-docs:check`. -- `llms.txt` is an AI index; keep it high-signal and avoid drift-prone, exhaustive inline API inventories. - -## Layout - -**TypeScript (`graphrefly-ts`) — cleave A executed 2026-05-15 (slices A1–A4); install-time model locked 2026-05-14:** - -Three published packages with an explicit substrate-vs-presentation split (see "Three-package install-time model" below). **Cleave A is DONE** — see `archive/docs/SESSION-DS-cleave-A-file-moves.md` for the file-move record + post-execution corrections. - -- Root `src/` — the **presentation package `@graphrefly/graphrefly`**. Post-cleave it owns the 4-layer structure (`base/ utils/ presets/ solutions/`) + `compat/`, and `src/index.ts` re-exports substrate from `@graphrefly/pure-ts` (peer) for ergonomic single-import UX. Substrate provider is chosen at install time: install `@graphrefly/pure-ts` (default) OR redirect to `@graphrefly/native` via npm/pnpm `overrides` (Q28 lock = option c). **⚠️ Native-drop-in NOT functional — RESOLVED 2026-05-15 as D206 (Option A + Option C follow-on).** The Q28/D198 overrides redirect does NOT work as written (`@graphrefly/native`'s napi surface is async-only — Core on a tokio blocking pool, sync calls deadlock per D070/D077 — while `@graphrefly/graphrefly` consumes pure-ts's sync API) and is **deferred pending D080**. Per **D206**: `@graphrefly/pure-ts` is the **sole working sync substrate** for `@graphrefly/graphrefly`; `@graphrefly/native` is an honest **async substrate** (`createNativeImpl()` wrapper shipped per Option C; bindings crate source at `0.1.0` per D265 hold-local — npm latest is `0.0.3` until the user-gated tag push triggers OIDC republish) + parity arm — **not** a sync drop-in for presentation via overrides; the async-everywhere presentation rebase (Option B / D080) stays deferred until consumer pressure. Canonical: `docs/rust-port-decisions.md` D206/D207 + `archive/docs/SESSION-DS-native-substrate-contract.md`. Legacy Phase-13.9.A shim folders (`src/{patterns,extra,core,graph,testing}/*`) were deleted — no backward-compat paths. -- `packages/pure-ts/src/` — the **pure-TS substrate implementation**. Permanent first-class peer alongside `@graphrefly/native` (and a future `@graphrefly/wasm` if a consumer surfaces; see Unit 6 note below). Substrate-only post-cleave: - - `core/` — message protocol, `node` primitive, batch, sugar constructors (Phase 0). `core/_internal/` holds substrate-internal utilities (`ring-buffer`, `sizeof`, `timer`/`ResettableTimer`) used by `graph/` + reactive structures. - - `graph/` — `Graph` container, describe/observe, snapshot (Phase 1+) - - `extra/` — operators, sync sources, `sources/event/timer` (`fromTimer`), data structures, storage (Node tiers), `composition/{stratify,topology-diff,pubsub}`, `sources/async` (`fromPromise`/`fromAsyncIter`/`fromAny`), `sources/_keepalive`. **Substrate-vs-presentation classification per `extra/` row is locked in `~/src/graphrefly-rs/CLAUDE.md` § "extra/ row classification"; post-execution corrections to A1 doc Q4/Q7/Q8 are recorded in `archive/docs/SESSION-DS-cleave-A-file-moves.md`.** - - `patterns/`, `compat/` — **removed from pure-ts.** All `patterns/*` are presentation (D193) and now live in `@graphrefly/graphrefly` (root `src/{utils,presets}/`); `compat/*` moved to root `src/compat/`. -- `packages/parity-tests/` — cross-impl parity scenarios (vitest `describe.each([pureTsImpl, rustImpl])` when a local `@graphrefly/native` `.node` is built or the published package is installed). See `packages/parity-tests/README.md` for the milestone registry + the "parity scenarios are the consumer pressure signal" rule (D196). **`packages/parity-tests/impls/types.ts` `Impl` interface IS the public-API contract** for the substrate peers (`@graphrefly/pure-ts` and `@graphrefly/native`) — widening it is a public API decision. -- `packages/cli` — workspace consumer of `@graphrefly/graphrefly`. Imports presentation (e.g. `SurfaceError`) from the root package barrel; substrate flows through the root shim's `export * from "@graphrefly/pure-ts"`. - -### Three-package install-time model (Unit 6 D198, locked 2026-05-14) - -| Package | Contains | Build artifact | Substrate or presentation? | -|---|---|---|---| -| `@graphrefly/pure-ts` | Full TS implementation of the Rust-portable substrate: `core/`, `graph/`, `extra/operators/`, `extra/sources/sync` + `fromTimer`, `extra/data-structures/`, `extra/storage/` (Node tiers), `extra/composition/stratify`. | TS only — browser + Node | substrate | -| `@graphrefly/native` | Rust impl of the same substrate via napi. Thin TS wrapper exposes the napi surface. | `.node` binary + TS wrapper | substrate (Node-only) | -| `@graphrefly/graphrefly` | The parts that **never go to Rust**: `patterns/*`, `extra/io/*`, `extra/composition/*` (except `stratify`), `extra/mutation/*`, `extra/sources/event` (`fromEvent`, `fromRaf`), browser sources, graph-sugar (`graph.log/list/map/index`), `compat/*`. | TS only | presentation | - -``` -@graphrefly/graphrefly ← presentation only - │ peerDependency: pick ONE substrate provider - ▼ -@graphrefly/pure-ts OR @graphrefly/native -``` - -Both substrate packages MUST expose the same public API — enforced by `packages/parity-tests/`. **No facade with runtime fallback**: the user picks at install time. Supersedes PART 13 Deferred 1's `optionalDependencies` facade plan. `@graphrefly/wasm` is deferred — adds when a browser-Rust consumer surfaces; until then `@graphrefly/pure-ts` is the universal fallback. - -Layering predicate that decides which package gets a new symbol lives in `~/src/graphrefly-rs/CLAUDE.md` § "Layering predicate — substrate vs presentation" (single source of truth, D193). - -### 4-layer model inside `@graphrefly/graphrefly` (Unit 8 D200, locked 2026-05-14) - -Strict top-down dependency layering (CI-enforced via `scripts/check-layer-boundary.ts`, wired into `pnpm lint`; D201 — mechanism amended 2026-05-15 from "Biome custom rule" to a zero-dep script since GritQL can't express rank comparison. Cleave-A layer-boundary residuals CLOSED 2026-05-15: `scripts/layer-boundary-baseline.json` baseline is now empty (`[]`) — the ratchet hard-fails ANY layer-boundary violation. Do not add baseline entries to silence new violations; fix the layering instead): - -| Layer | Charter | Examples | -|---|---|---| -| `base/` | **Domain-agnostic infrastructure.** Helpers with NO domain semantics. | io (http/ws/sse/webhook), composition helpers (verifiable, distill, pubsub, backpressure, externalProducer), mutation wrappers (lightMutation, auditLog), worker bridge, browser/runtime sources (fromEvent, fromRaf, fromGitHook, fromFSWatch), meta (domainMeta, keepalive) | -| `utils/` | **Domain building blocks.** Single-purpose factories returning a `Node` or `Graph` (was consolidation-plan's "building blocks"). | messaging (topic, subscription, hub, topicBridge), orchestration (pipelineGraph, approvalGate, humanInput, tracker, classify, catch), cqrs, reduction, memory, ai/{prompts, agents, safety, extractors, adapters}, inspect, harness (stage types, evalSource, beforeAfterCompare) | -| `presets/` | **Opinionated compositions of utils** (≥3 utils typically). Single-factory products. Vocabulary preserved from consolidation plan. | agentLoop, agentMemory, resilientPipeline, harnessLoop, refineLoop, spawnable, inspect (composite), guardedExecution, reactiveFactStore, taggedContextPool, heterogeneousDebate, actorPool | -| `solutions/` | **User-facing packaged products.** Top-level barrel re-exports presets + per-vertical multi-preset starter kits (D202 = (c) both). | `solutions/index.ts` barrel re-exports; vertical folders (`solutions/customer-support-bot/`, `solutions/code-review-agent/`, etc.) deferred until consumer pressure | -| `compat/` | External framework adapters (NestJS, React, Vue, Solid, Svelte, ag-ui translator, a2ui). | sits alongside the 4 layers; depends on solutions/presets/utils/base in top-down order | - -Dependency rules: - -``` -substrate (@graphrefly/pure-ts | @graphrefly/native) - ▲ - │ -base/ (no domain semantics) - ▲ - │ -utils/ (domain building blocks) - ▲ - │ -presets/ (opinionated compositions of utils) - ▲ - │ -solutions/ (user-facing packaged products) - ▲ - │ -compat/ (external framework adapters) -``` - -Within a layer: free composition (e.g., `utils/orchestration/human-input.ts` may import `utils/messaging/topic.ts` — both utils). Cross-layer: strictly top-down. Circular within-layer rejected. Layer-placement rubric: "zero domain → base; single-domain primitive returning Node/Graph → utils; ≥3 utils composition → preset; ≥2 presets or full vertical with adapters/storage wiring → solution." - -Source: `archive/docs/SESSION-rust-port-layer-boundary.md` Units 6, 8 (user-locked 2026-05-14). - -### Browser / Node / Universal subpath convention (TS) - -Public TS APIs are split into three tiers so browser and Node consumers pull only runnable code: - -- **Universal default** (`@graphrefly/graphrefly`, `@graphrefly/graphrefly/extra`, `@graphrefly/graphrefly/utils/`) — browser + Node safe. Zero `node:*` imports, zero DOM globals. -- **Node-only** (`@graphrefly/graphrefly/extra/node`, `@graphrefly/graphrefly/utils//node`) — may import `node:*`. Use for `fileStorage`, `sqliteStorage`, `fromGitHook`, `fromFSWatch`, the node `fallbackAdapter` variant, etc. -- **Browser-only** (`@graphrefly/graphrefly/extra/browser`, `@graphrefly/graphrefly/utils//browser`) — may use DOM globals. Use for `indexedDbStorage`, `webllmAdapter`, `chromeNanoAdapter`, browser cascade presets. - -The build enforces this via `assertBrowserSafeBundles` in `packages/pure-ts/tsup.config.ts` `onSuccess` — any universal entry that transitively imports a Node builtin fails the build with a `via X → Y → Z` chain. Adding a new subpath requires updating BOTH `packages/pure-ts/tsup.config.ts` `ENTRY_POINTS` (+ `nodeOnlyEntries` when Node-only) AND `packages/pure-ts/package.json` `exports`, then mirroring the entry in the root shim (`tsup.config.ts` + `package.json` `exports` + a one-liner `src/.ts`). See `docs/docs-guidance.md` § "Browser / Node / Universal split" for the full convention. - -**Python (`graphrefly-py`):** -- `src/graphrefly/core/` — message protocol, `node` primitive, batch, sugar constructors (Phase 0) -- `src/graphrefly/graph/` — `Graph` container, describe/observe, snapshot (Phase 1+) -- `src/graphrefly/extra/` — operators, sources, data structures, resilience (Phase 2–3) -- `src/graphrefly/patterns/` — domain-layer APIs: orchestration, messaging, memory, AI, CQRS, reactive layout (Phase 4+) -- `src/graphrefly/compat/` — async runners: asyncio, trio (Phase 5+) -- `src/graphrefly/integrations/` — framework integrations: FastAPI (Phase 5+) - -## Design invariants (spec §5.8–5.12) - -These are non-negotiable across all implementations. Validate every change against them. - -*Summary; canonical text in `~/src/graphrefly/GRAPHREFLY-SPEC.md` §5.8–5.12. Treat that as the authority if anything below disagrees.* - -1. **No polling.** State changes propagate reactively via messages. Never poll a node's value on a timer or busy-wait for status. Use reactive timer sources (`fromTimer`/`from_timer`, `fromCron`/`from_cron`) instead. -2. **No imperative triggers.** All coordination uses reactive `NodeInput` signals and message flow through topology. No event emitters, callbacks, or `setTimeout`/`threading.Timer` + `set()` workarounds. If you need a trigger, it's a reactive source node. -3. **No raw async primitives in the reactive layer.** TS: no bare `Promise`, `queueMicrotask`, `setTimeout`, or `process.nextTick`. PY: no bare `asyncio.ensure_future`, `asyncio.create_task`, `threading.Timer`, or raw coroutines. Async boundaries belong in sources (`fromPromise`/`from_awaitable`, `fromAsyncIter`/`from_async_iter`) and the runner layer, not in node fns or operators. -4. **Central timer and `messageTier`/`message_tier` utilities.** TS: use `clock.ts` for all timestamps. PY: use `clock.py`. Use `messageTier`/`message_tier` utilities for tier classification — never hardcode type checks for checkpoint or batch gating. -5. **Phase 4+ APIs must be developer-friendly.** Domain-layer APIs (orchestration, messaging, memory, AI, CQRS) use sensible defaults, minimal boilerplate, and clear errors. Protocol internals (`DIRTY`, `RESOLVED`, bitmask) never surface in primary APIs — accessible via `.node()` or `inner` when needed. - -## Time utility rule - -- **TS:** all timestamps go through `src/core/clock.ts`. Internal/event-order durations: `monotonicNs()`. Wall-clock attribution: `wallClockNs()`. -- **PY:** same rule with `src/graphrefly/core/clock.py`. Functions: `monotonic_ns()` and `wall_clock_ns()`. - -## Auto-checkpoint trigger rule - -- For persistence auto-checkpoint behavior, gate saves by `messageTier`/`message_tier >= 3`. -- Do not describe this as DATA/RESOLVED-only; terminal/teardown lifecycle tiers are included. - -## Debugging composition (mandatory procedure) - -When debugging OOM, infinite loops, silent failures, or unexpected values in composed factories, follow the **"Debugging composition"** section in `~/src/graphrefly/COMPOSITION-GUIDE.md`. That is the single source of truth for the procedure. Do not skip or improvise around it. - -## Dry-run equivalence rule - -**Dry-run must be behaviorally identical to the real run except for the actual LLM wire call.** Every observability surface the real run exercises — stage trace, budget stream, `graph.describe` (incl. `describe({ explain })` causal-chain mode), `observe`, stats readouts — must also be exercised in dry-run on the same graph topology. Regressions in `describe` / `describe({ explain })` / `observe` or in graph wiring must surface in dry-run BEFORE the user pays for a real run. - -When building an example or demo that has a dry-run path: -- Construct the exact same graph as the real run; only the adapter differs (shipped `dryRunAdapter` or a shaped mock swapped in via `withDryRun`). -- Call every inspection / explainability method the real run calls. If the real run prints a causal chain, so must dry-run. If the real run subscribes to `budget.totals`, so must dry-run (totals at zero is fine — presence is the point). -- On regression, exit non-zero from dry-run with a diagnostic so the user sees the bug *before* the confirmation prompt. -- Inspection tools to reach for first (all shipped — note: there is **no** `graph.explain()` method; causal chains are `describe({ explain })`, and formats are pure renderers over a describe snapshot, not a `describe({ format })` option): `graph.describe()` then render via `graphSpecToPretty` / `graphSpecToMermaid` / `graphSpecToD2` from `@graphrefly/graphrefly/extra/render`; `graph.describe({ explain: { from, to } })`; `graph.observe(path)`; `reachable(graph, from)`; `graphProfile(graph)`; `harnessProfile(graph)`. If you need a new inspection tool that isn't in this list, flag it in `docs/optimizations.md` as a library candidate before shipping ad-hoc scripts. - -## Python-specific invariants - -- **Thread safety:** Design for GIL and free-threaded Python. Per-subgraph `RLock`, per-node `_cache_lock`. Core APIs documented as thread-safe (see roadmap Phase 0.4). -- **No `async def` / `Awaitable` in public APIs.** All public functions return `Node[T]`, `Graph`, `None`, or a plain synchronous value. -- **Diamond resolution** via unlimited-precision Python `int` bitmask (TS uses `Uint32Array` + `BigInt` for fan-in >31). -- **Context managers:** PY uses `with batch():` instead of TS's `batch(() => ...)`. -- **`|` pipe operator:** PY `Node.__or__` maps to TS `pipe()`. - -## Claude skills (workflows) - -Project-local skills live under `.claude/skills/`. These skills operate on **both** TS and PY repos when relevant: - -- **dev-dispatch** — plan, align with spec, implement, self-test -- **qa** — adversarial review, fixes, test + lint + build, doc touch-ups -- **design-review** — Q5–Q9 design lens (abstraction, long-term shape, reactive composability, alternatives, coverage). Use BEFORE coding for new primitives; complementary to `/qa` (which finds bugs in landed code). -- **parity** — cross-language parity check (TS vs PY) +Project-local skills under `.claude/skills/`: -Invoke via the user's Claude Code slash commands or skill names when relevant. +- **decision-guard** — recall locked D#/values/floor before any decision question. +- **spec-amend** — spec-first protocol amendment (rules + TLA+ + conformance, then code). +- **conformance** — drive behavioral conformance scenarios green per runtime. +- **dashboard** — build / check the `~/src/graphrefly` docs dashboard + consistency gate. +- **dev-dispatch** — plan, align with spec, implement, self-test. +- **qa** — adversarial review, fixes, test + lint + build, doc touch-ups. +- **design-review** — Q5–Q9 design lens before coding new primitives. From 6ddd6ac2e64568e78d4daf8489e139a6081674b9 Mon Sep 17 00:00:00 2001 From: David Chen Date: Thu, 28 May 2026 17:59:19 -0700 Subject: [PATCH 002/175] feat(ts): introduce @graphrefly/ts package with TypeScript support - Added new package `@graphrefly/ts` with its own `package.json`, `tsconfig.json`, and build configuration using `tsup`. - Updated `package.json` to include new test scripts for the `@graphrefly/ts` package. - Enhanced the testing framework with Vitest for the new package, including multiple test files for core functionality and conformance. - Updated `pnpm-lock.yaml` to reflect new dependencies for TypeScript and testing tools. - Implemented core functionality in `src/index.ts` and added various test cases to ensure expected behavior. --- package.json | 3 +- packages/ts/package.json | 37 + .../ts/src/__tests__/batch-dynamic.test.ts | 100 +++ packages/ts/src/__tests__/conformance.test.ts | 156 ++++ packages/ts/src/__tests__/control.test.ts | 127 +++ packages/ts/src/__tests__/core.test.ts | 176 ++++ packages/ts/src/__tests__/lifecycle.test.ts | 169 ++++ packages/ts/src/__tests__/qa-fixes.test.ts | 63 ++ packages/ts/src/batch/batch.ts | 92 ++ packages/ts/src/ctx/types.ts | 63 ++ packages/ts/src/dispatcher/index.ts | 103 +++ packages/ts/src/index.ts | 19 + packages/ts/src/node/node.ts | 805 ++++++++++++++++++ packages/ts/src/protocol/messages.ts | 73 ++ packages/ts/tsconfig.json | 17 + packages/ts/tsup.config.ts | 9 + packages/ts/vitest.config.ts | 7 + pnpm-lock.yaml | 12 + 18 files changed, 2030 insertions(+), 1 deletion(-) create mode 100644 packages/ts/package.json create mode 100644 packages/ts/src/__tests__/batch-dynamic.test.ts create mode 100644 packages/ts/src/__tests__/conformance.test.ts create mode 100644 packages/ts/src/__tests__/control.test.ts create mode 100644 packages/ts/src/__tests__/core.test.ts create mode 100644 packages/ts/src/__tests__/lifecycle.test.ts create mode 100644 packages/ts/src/__tests__/qa-fixes.test.ts create mode 100644 packages/ts/src/batch/batch.ts create mode 100644 packages/ts/src/ctx/types.ts create mode 100644 packages/ts/src/dispatcher/index.ts create mode 100644 packages/ts/src/index.ts create mode 100644 packages/ts/src/node/node.ts create mode 100644 packages/ts/src/protocol/messages.ts create mode 100644 packages/ts/tsconfig.json create mode 100644 packages/ts/tsup.config.ts create mode 100644 packages/ts/vitest.config.ts diff --git a/package.json b/package.json index b48861de..43a3ae11 100644 --- a/package.json +++ b/package.json @@ -552,8 +552,9 @@ "build": "pnpm --filter @graphrefly/pure-ts build && NODE_OPTIONS=--max-old-space-size=8192 tsup", "build:shim": "NODE_OPTIONS=--max-old-space-size=8192 tsup", "prepare": "node -e \"const fs=require('fs');const distOk=fs.existsSync('dist/index.js')&&fs.existsSync('packages/pure-ts/dist/index.js');const src=fs.existsSync('tsup.config.ts');process.exit(distOk||!src?0:1)\" || pnpm run build", - "test": "pnpm --filter @graphrefly/pure-ts test && pnpm test:graphrefly && pnpm --filter @graphrefly/parity-tests test", + "test": "pnpm --filter @graphrefly/ts test && pnpm --filter @graphrefly/pure-ts test && pnpm test:graphrefly && pnpm --filter @graphrefly/parity-tests test", "test:graphrefly": "vitest run", + "test:ts": "pnpm --filter @graphrefly/ts test", "test:pure-ts": "pnpm --filter @graphrefly/pure-ts test", "test:parity": "pnpm --filter @graphrefly/parity-tests test", "test:hermes": "pnpm --filter @graphrefly/pure-ts build && node scripts/hermes-smoke/run.mjs", diff --git a/packages/ts/package.json b/packages/ts/package.json new file mode 100644 index 00000000..7c16d281 --- /dev/null +++ b/packages/ts/package.json @@ -0,0 +1,37 @@ +{ + "name": "@graphrefly/ts", + "version": "0.0.0", + "description": "GraphReFly clean-slate TypeScript substrate — node / dispatcher / pool / wave protocol.", + "type": "module", + "sideEffects": false, + "main": "dist/index.cjs", + "module": "dist/index.js", + "types": "dist/index.d.ts", + "exports": { + ".": { + "import": { + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + }, + "require": { + "types": "./dist/index.d.cts", + "default": "./dist/index.cjs" + } + } + }, + "files": ["dist", "LICENSE"], + "publishConfig": { + "access": "public" + }, + "scripts": { + "build": "tsup", + "test": "vitest run", + "test:watch": "vitest" + }, + "license": "MIT", + "devDependencies": { + "tsup": "^8.5.1", + "typescript": "^5.7.0", + "vitest": "^3.0.0" + } +} diff --git a/packages/ts/src/__tests__/batch-dynamic.test.ts b/packages/ts/src/__tests__/batch-dynamic.test.ts new file mode 100644 index 00000000..11f70961 --- /dev/null +++ b/packages/ts/src/__tests__/batch-dynamic.test.ts @@ -0,0 +1,100 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { batch, dynamicNode, node } from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const types = (msgs: Message[]) => msgs.map((m) => m[0]); + +describe("batch (R-batch-coalesce / D12)", () => { + it("coalesces a diamond join to ONE recompute when both sources change", () => { + let fires = 0; + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 1 }); + const d = node([a, b], (ctx: Ctx) => { + fires++; + ctx.down([ + ["DATA", (ctx.depRecords[0].latest as number) + (ctx.depRecords[1].latest as number)], + ]); + }); + collect(d); + expect(d.cache).toBe(2); + + fires = 0; + batch(() => { + a.down([["DATA", 10]]); + b.down([["DATA", 20]]); + }); + expect(fires).toBe(1); // one recompute, not two + expect(d.cache).toBe(30); + }); + + it("defers DATA to commit: downstream sees DIRTY during the batch, DATA after", () => { + const a = node([], null, { initial: 1 }); + const { msgs } = collect(a); + msgs.length = 0; + batch(() => { + a.down([["DATA", 9]]); + // inside the batch: DIRTY emitted, DATA still deferred + expect(types(msgs)).toEqual(["DIRTY"]); + expect(a.cache).toBe(1); // not yet committed + }); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(a.cache).toBe(9); + }); + + it("rollback on throw discards deferred emissions (downstream un-dirties)", () => { + const a = node([], null, { initial: 1 }); + const { msgs } = collect(a); + msgs.length = 0; + expect(() => + batch(() => { + a.down([["DATA", 99]]); + throw new Error("abort"); + }), + ).toThrow("abort"); + expect(a.cache).toBe(1); // unchanged + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); // dirty balanced by resolved + }); + + it("bctx.rollback() is the explicit escape hatch", () => { + const a = node([], null, { initial: 1 }); + collect(a); + batch((bctx) => { + a.down([["DATA", 50]]); + bctx.rollback(); + }); + expect(a.cache).toBe(1); + }); +}); + +describe("dynamicNode (R-dynamic-node / D35)", () => { + it("reads a selected dep; an unread dep's change is absorbed (no downstream DATA)", () => { + const sel = node<"a" | "b">([], null, { initial: "a" }); + const a = node([], null, { initial: 100 }); + const b = node([], null, { initial: 200 }); + const router = dynamicNode([sel, a, b], (ctx: Ctx) => { + const which = ctx.track?.(0) as "a" | "b"; + const value = which === "a" ? (ctx.track?.(1) as number) : (ctx.track?.(2) as number); + ctx.down([["DATA", value]]); + }); + const { msgs } = collect(router); + expect(router.cache).toBe(100); // sel="a" -> reads a + msgs.length = 0; + + // change the UNREAD dep b -> fn fires, output unchanged (still 100) -> equals absorbs + b.down([["DATA", 999]]); + expect(router.cache).toBe(100); + expect(types(msgs)).not.toContain("DATA"); + expect(types(msgs)).toContain("RESOLVED"); + + // change the READ dep a -> downstream DATA + msgs.length = 0; + a.down([["DATA", 111]]); + expect(router.cache).toBe(111); + expect(msgs).toContainEqual(["DATA", 111]); + }); +}); diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts new file mode 100644 index 00000000..09222f42 --- /dev/null +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -0,0 +1,156 @@ +/** + * Behavioral conformance — TS arm of ~/src/graphrefly/spec/conformance.jsonl (D24). + * + * Each test is the TS adapter for a language-agnostic scenario: it builds the scenario's + * topology, drives its input wave sequence, and asserts the expected OBSERVABLE wave output. + * + * C-1 (cross-graph diamond) is NOT here — it requires the wire bridge (backlog B2). The + * in-process diamond core it leans on is green in core.test.ts (R-diamond/R-two-phase). + */ + +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { node } from "../index.js"; + +const types = (msgs: Message[]) => msgs.map((m) => m[0]); +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} + +describe("C-2 async-result arriving at paused node (R-async-paused, R-pause-lockset)", () => { + it("buffers the async result while paused, replays it on final-lock RESUME", () => { + let cctx: Ctx | null = null; + const trigger = node([], null, { initial: 0 }); + // async-pool node: the fn stashes its ctx and resolves later (simulated async). + const n = node( + [trigger], + (ctx: Ctx) => { + cctx = ctx; + }, + { pool: "async" }, + ); + const { msgs } = collect(n); + expect(cctx).not.toBeNull(); // fn ran on activation, no emit yet + + const L = Symbol("pause"); + n.up([["PAUSE", L]]); + + msgs.length = 0; + // async result resolves WHILE paused -> buffered, not delivered (DR-3). + (cctx as Ctx).down([["DATA", 42]]); + expect(msgs).toEqual([]); + expect(n.cache).toBeUndefined(); + + n.up([["RESUME", L]]); // final-lock RESUME -> replay the buffered settle slice + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(msgs.at(-1)).toEqual(["DATA", 42]); + expect(n.cache).toBe(42); + }); +}); + +describe("C-3 INVALIDATE × ctx.state × onInvalidate (R-invalidate-idempotent, R-ctx-state)", () => { + it("cascades once, fires onInvalidate, preserves ctx.state, resets dep prevData", () => { + const statesAtRun: unknown[] = []; + let onInv = 0; + const s = node([], null, { initial: 1 }); + const d = node([s], (ctx: Ctx) => { + statesAtRun.push(ctx.state.get()); // prior state visible at run time + ctx.state.set("kept"); + ctx.onInvalidate(() => { + onInv++; + }); + ctx.down([["DATA", (ctx.depRecords[0].latest as number) * 2]]); + }); + const { msgs } = collect(d); + expect(d.cache).toBe(2); + expect(statesAtRun).toEqual([undefined]); // first run: fresh state + + msgs.length = 0; + s.down([["INVALIDATE"]]); + expect(types(msgs)).toEqual(["INVALIDATE"]); // cascaded downstream exactly once + expect(onInv).toBe(1); + expect(d.cache).toBeUndefined(); + expect(d.status).toBe("sentinel"); + + // idempotent: a second INVALIDATE on an already-reset upstream is a no-op + msgs.length = 0; + s.down([["INVALIDATE"]]); + expect(msgs).toEqual([]); + expect(onInv).toBe(1); + + // ctx.state preserved across INVALIDATE (lifecycle-continue, NOT fresh-lifecycle) + s.down([["DATA", 5]]); + expect(statesAtRun).toEqual([undefined, "kept"]); + expect(d.cache).toBe(10); + }); +}); + +describe("C-4 mixed sync/async diamond (R-diamond, R-two-phase, R-first-run-gate)", () => { + it("joins exactly once after BOTH the sync and async legs settle", () => { + let dRuns = 0; + let cctx: Ctx | null = null; + const a = node([], null, { initial: 1 }); + const b = node([a], (ctx: Ctx) => + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 10]]), + ); // sync leg + const c = node( + [a], + (ctx: Ctx) => { + cctx = ctx; // async leg: defer the emit + }, + { pool: "async" }, + ); + const d = node([b, c], (ctx: Ctx) => { + dRuns++; + ctx.down([ + ["DATA", (ctx.depRecords[0].latest as number) + (ctx.depRecords[1].latest as number)], + ]); + }); + + collect(d); + // b settled synchronously (11); the async leg is deferred -> first-run gate holds d + expect(dRuns).toBe(0); + expect(d.cache).toBeUndefined(); + + (cctx as Ctx).down([["DATA", 21]]); // async leg resolves + expect(dRuns).toBe(1); // joined exactly once + expect(d.cache).toBe(32); // 11 + 21 + }); +}); + +describe("C-5 PAUSE lockset multi-source (R-pause-lockset, R-pause-modes)", () => { + it("stays paused until every lock RESUMEs; dup PAUSE + unknown RESUME are no-ops", () => { + let runs = 0; + const s = node([], null, { initial: 0 }); + const n = node([s], (ctx: Ctx) => { + runs++; + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + collect(n); + expect(n.cache).toBe(0); + runs = 0; + + const LA = Symbol("A"); + const LB = Symbol("B"); + n.up([["PAUSE", LA]]); + n.up([["PAUSE", LB]]); + n.up([["PAUSE", LA]]); // duplicate -> idempotent (lockset) + + s.down([["DATA", 1]]); // dep changes while paused + expect(runs).toBe(0); // fn held + expect(n.cache).toBe(0); + + n.up([["RESUME", LA]]); // release A — LB still held + expect(runs).toBe(0); // STILL paused + expect(n.cache).toBe(0); + + n.up([["RESUME", Symbol("unknown")]]); // unknown id -> no-op + expect(runs).toBe(0); + + n.up([["RESUME", LB]]); // last lock released -> resume, fire once with latest + expect(runs).toBe(1); + expect(n.cache).toBe(1); + }); +}); diff --git a/packages/ts/src/__tests__/control.test.ts b/packages/ts/src/__tests__/control.test.ts new file mode 100644 index 00000000..092dc5c1 --- /dev/null +++ b/packages/ts/src/__tests__/control.test.ts @@ -0,0 +1,127 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { node } from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const types = (msgs: Message[]) => msgs.map((m) => m[0]); + +describe("PAUSE/RESUME lockset (R-pause-lockset, R-pause-modes default)", () => { + it("multi-source lockset: releasing one lock does not resume while another holds (C-5)", () => { + const a = node([], null, { initial: 1 }); + let runs = 0; + const d = node([a], (ctx: Ctx) => { + runs++; + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 1]]); + }); + collect(d); + expect(d.cache).toBe(2); + + const LA = Symbol("LA"); + const LB = Symbol("LB"); + d.up([["PAUSE", LA]]); + d.up([["PAUSE", LA]]); // same-id repeat is idempotent + d.up([["PAUSE", LB]]); + + runs = 0; + a.down([["DATA", 10]]); // dep wave while paused -> default mode skips fn + expect(runs).toBe(0); + expect(d.cache).toBe(2); // not recomputed yet + + d.up([["RESUME", LA]]); // LB still held -> stay paused + expect(runs).toBe(0); + d.up([["RESUME", Symbol("unknown")]]); // unknown id -> no-op + expect(runs).toBe(0); + + d.up([["RESUME", LB]]); // final release -> fire once with latest + expect(runs).toBe(1); + expect(d.cache).toBe(11); + }); + + it("pausable:false ignores PAUSE/RESUME (timer-source semantics)", () => { + const a = node([], null, { initial: 1 }); + let runs = 0; + const d = node( + [a], + (ctx: Ctx) => { + runs++; + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 1]]); + }, + { pausable: false }, + ); + collect(d); + runs = 0; + d.up([["PAUSE", Symbol()]]); + a.down([["DATA", 5]]); + expect(runs).toBe(1); // not gated + expect(d.cache).toBe(6); + }); +}); + +describe("async pool (R-sync-core async label, R8 late-emit pairing)", () => { + it("a late ctx.down from an async fn pairs DIRTY+DATA downstream", () => { + const a = node([], null, { initial: 3 }); + let resolve: (() => void) | null = null; + const dbl = node( + [a], + (ctx: Ctx) => { + const v = ctx.depRecords[0].latest as number; + resolve = () => ctx.down([["DATA", v * 2]]); + }, + { pool: "async" }, + ); + const { msgs } = collect(dbl); + // fn ran synchronously (set resolve) but emitted nothing yet. + expect(dbl.cache).toBeUndefined(); + expect(types(msgs)).toEqual(["START"]); + + resolve?.(); + expect(dbl.cache).toBe(6); + expect(types(msgs)).toEqual(["START", "DIRTY", "DATA"]); + }); +}); + +describe("async-result at a paused node (R-async-paused / C-2)", () => { + it("buffers the result while paused, replays on final RESUME", () => { + const a = node([], null, { initial: 3 }); + let resolve: (() => void) | null = null; + const an = node( + [a], + (ctx: Ctx) => { + const v = ctx.depRecords[0].latest as number; + resolve = () => ctx.down([["DATA", v * 2]]); + }, + { pool: "async" }, + ); + const { msgs } = collect(an); + msgs.length = 0; + + const L = Symbol("L"); + an.up([["PAUSE", L]]); + resolve?.(); // async result arrives while paused -> buffered, not delivered + expect(an.cache).toBeUndefined(); + expect(types(msgs)).toEqual([]); + + an.up([["RESUME", L]]); // final RESUME -> replay buffered result + expect(an.cache).toBe(6); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + }); +}); + +describe("replayBuffer (R-replay-buffer)", () => { + it("late subscriber receives the last N DATA after START", () => { + const s = node([], null, { replayBuffer: 3 }); + collect(s); + s.down([["DATA", 1]]); + s.down([["DATA", 2]]); + s.down([["DATA", 3]]); + s.down([["DATA", 4]]); + + const { msgs } = collect(s); + expect(types(msgs)).toEqual(["START", "DATA", "DATA", "DATA"]); + expect(msgs.slice(1).map((m) => m[1])).toEqual([2, 3, 4]); + }); +}); diff --git a/packages/ts/src/__tests__/core.test.ts b/packages/ts/src/__tests__/core.test.ts new file mode 100644 index 00000000..e5b384f4 --- /dev/null +++ b/packages/ts/src/__tests__/core.test.ts @@ -0,0 +1,176 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { node } from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} + +const types = (msgs: Message[]) => msgs.map((m) => m[0]); + +describe("state node (manual source)", () => { + it("push-on-subscribe delivers START then cached DATA (R-push-subscribe, R-initial)", () => { + const s = node([], null, { initial: 5 }); + const { msgs } = collect(s); + expect(msgs).toEqual([["START"], ["DATA", 5]]); + expect(s.cache).toBe(5); + expect(s.status).toBe("settled"); + }); + + it("uncached subscribe delivers only START", () => { + const s = node([], null); + const { msgs } = collect(s); + expect(msgs).toEqual([["START"]]); + expect(s.cache).toBeUndefined(); + expect(s.status).toBe("sentinel"); + }); + + it("external down emits DIRTY before DATA (R-dirty-before-data, two-phase)", () => { + const s = node([], null, { initial: 1 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["DATA", 2]]); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(msgs[1]).toEqual(["DATA", 2]); + expect(s.cache).toBe(2); + }); + + it("null is a valid DATA value (R-data-payload)", () => { + const s = node([], null, { initial: null }); + const { msgs } = collect(s); + expect(msgs).toEqual([["START"], ["DATA", null]]); + expect(s.cache).toBeNull(); + }); +}); + +describe("equals -> RESOLVED (R-equals)", () => { + it("re-emitting the same value yields RESOLVED, not DATA; cache unchanged", () => { + const s = node([], null, { initial: 5 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["DATA", 5]]); + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); + expect(s.cache).toBe(5); + }); + + it("a changed value yields DATA", () => { + const s = node([], null, { initial: 5 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["DATA", 6]]); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(s.cache).toBe(6); + }); + + it("does NOT substitute on a multi-DATA wave (R-equals exclusivity)", () => { + const s = node([], null, { initial: 5 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([ + ["DATA", 5], + ["DATA", 5], + ]); + // dataCount > 1 => no equals substitution; both pass as DATA. + expect(types(msgs)).toEqual(["DIRTY", "DATA", "DATA"]); + }); +}); + +describe("compute node (derived)", () => { + it("computes from a dep and recomputes on change", () => { + const count = node([], null, { initial: 2 }); + const doubled = node([count], (ctx: Ctx) => { + ctx.down([["DATA", (ctx.depRecords[0].latest as number) * 2]]); + }); + const { msgs } = collect(doubled); + expect(doubled.cache).toBe(4); + expect(msgs).toContainEqual(["DATA", 4]); + + msgs.length = 0; + count.down([["DATA", 10]]); + expect(doubled.cache).toBe(20); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(msgs[1]).toEqual(["DATA", 20]); + }); +}); + +describe("diamond (R-diamond, glitch-free join)", () => { + it("join node computes exactly once per upstream change, after both deps settle", () => { + let fireCount = 0; + const a = node([], null, { initial: 1 }); + const b = node([a], (ctx: Ctx) => + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 10]]), + ); + const c = node([a], (ctx: Ctx) => + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 20]]), + ); + const d = node([b, c], (ctx: Ctx) => { + fireCount++; + ctx.down([ + ["DATA", (ctx.depRecords[0].latest as number) + (ctx.depRecords[1].latest as number)], + ]); + }); + + collect(d); + expect(d.cache).toBe(32); // (1+10) + (1+20) + expect(fireCount).toBe(1); // joined once, not once per leg + + fireCount = 0; + a.down([["DATA", 2]]); + expect(d.cache).toBe(34); // (2+10) + (2+20) + expect(fireCount).toBe(1); // recomputed exactly once + }); +}); + +describe("first-run gate (R-first-run-gate)", () => { + it("partial:false holds fn until every dep has settled", () => { + let fired = false; + const a = node([], null, { initial: 1 }); + const b = node([], null); // uncached — never settles + node([a, b], () => { + fired = true; + }); + collect(node([a, b], () => {})); // force-activate a separate gate path + + // Build the gated node explicitly and activate it: + fired = false; + const gated = node([a, b], () => { + fired = true; + }); + gated.subscribe(() => {}); + expect(fired).toBe(false); // b never delivered real DATA -> gate holds + b.down([["DATA", 9]]); + expect(fired).toBe(true); // now both settled -> fires once + }); + + it("partial:true fires without waiting for all deps", () => { + let fired = false; + const a = node([], null, { initial: 1 }); + const b = node([], null); // uncached + const g = node( + [a, b], + (ctx: Ctx) => { + fired = true; + // fn body guards SENTINEL per dep (R-first-run-gate partial contract) + const bv = ctx.depRecords[1].latest; + ctx.down([["DATA", bv === undefined ? -1 : (bv as number)]]); + }, + { partial: true }, + ); + g.subscribe(() => {}); + expect(fired).toBe(true); + expect(g.cache).toBe(-1); + }); +}); + +describe("ctx.up direction guard (R-ctx-up)", () => { + it("throws on a down-only tier", () => { + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx: Ctx) => ctx.down([["DATA", 1]])); + d.subscribe(() => {}); + expect(() => d.up([["DATA", 5]])).toThrow(/down-only/); + expect(() => d.up([["COMPLETE"]])).toThrow(/down-only/); + expect(() => d.up([["INVALIDATE"]])).not.toThrow(); + }); +}); diff --git a/packages/ts/src/__tests__/lifecycle.test.ts b/packages/ts/src/__tests__/lifecycle.test.ts new file mode 100644 index 00000000..ecf2199a --- /dev/null +++ b/packages/ts/src/__tests__/lifecycle.test.ts @@ -0,0 +1,169 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { node } from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const types = (msgs: Message[]) => msgs.map((m) => m[0]); + +describe("terminal (R-terminal, R-deps-terminal)", () => { + it("auto-COMPLETE when ALL deps complete", () => { + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx: Ctx) => + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 1]]), + ); + const { msgs } = collect(d); + msgs.length = 0; + a.down([["COMPLETE"]]); + expect(types(msgs)).toEqual(["COMPLETE"]); + expect(d.status).toBe("completed"); + }); + + it("requires ALL deps complete, not ANY", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 2 }); + const d = node([a, b], (ctx: Ctx) => + ctx.down([ + ["DATA", (ctx.depRecords[0].latest as number) + (ctx.depRecords[1].latest as number)], + ]), + ); + const { msgs } = collect(d); + msgs.length = 0; + a.down([["COMPLETE"]]); + expect(types(msgs)).not.toContain("COMPLETE"); // only one dep complete + b.down([["COMPLETE"]]); + expect(types(msgs)).toContain("COMPLETE"); // now all complete + expect(d.status).toBe("completed"); + }); + + it("auto-ERROR when any dep errors (errorWhenDepsError)", () => { + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx: Ctx) => ctx.down([["DATA", 1]])); + const { msgs } = collect(d); + msgs.length = 0; + const err = new Error("boom"); + a.down([["ERROR", err]]); + expect(msgs).toContainEqual(["ERROR", err]); + expect(d.status).toBe("errored"); + }); + + it("ERROR with undefined payload is rejected (R-data-payload)", () => { + const s = node([], null, { initial: 1 }); + collect(s); + expect(() => s.down([["ERROR", undefined]])).toThrow(/non-SENTINEL/); + }); + + it("non-resubscribable terminal rejects late subscribe (R2.2.7.b)", () => { + const s = node([], null, { initial: 1 }); + collect(s); + s.down([["COMPLETE"]]); + expect(() => s.subscribe(() => {})).toThrow(/non-resubscribable/); + }); + + it("resubscribable terminal resets on late subscribe (R2.2.7.a)", () => { + const s = node([], null, { initial: 1, resubscribable: true }); + collect(s); + s.down([["COMPLETE"]]); + expect(s.status).toBe("completed"); + const { msgs } = collect(s); // resets, then push-on-subscribe + expect(s.status).toBe("settled"); + expect(msgs).toContainEqual(["DATA", 1]); + }); +}); + +describe("INVALIDATE (R-invalidate-idempotent, R-cleanup-hooks)", () => { + it("clears cache, fires onInvalidate, cascades — and is idempotent", () => { + const a = node([], null, { initial: 5 }); + let flushed = 0; + const d = node([a], (ctx: Ctx) => { + ctx.onInvalidate(() => flushed++); + ctx.down([["DATA", (ctx.depRecords[0].latest as number) * 2]]); + }); + const { msgs } = collect(d); + expect(d.cache).toBe(10); + msgs.length = 0; + + a.down([["INVALIDATE"]]); + expect(flushed).toBe(1); + expect(d.cache).toBeUndefined(); + expect(types(msgs)).toContain("INVALIDATE"); + + // second INVALIDATE: cache already reset -> no-op (no second flush/cascade) + msgs.length = 0; + a.down([["INVALIDATE"]]); + expect(flushed).toBe(1); + expect(types(msgs)).not.toContain("INVALIDATE"); + }); + + it("never-populated INVALIDATE is a no-op", () => { + const s = node([], null); // no cache + const { msgs } = collect(s); + msgs.length = 0; + s.down([["INVALIDATE"]]); + expect(msgs).toEqual([]); + }); +}); + +describe("same-wave merge (R-same-wave-merge)", () => { + it("DATA + INVALIDATE: cache advances then clears", () => { + const s = node([], null, { initial: 1 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["DATA", 9], ["INVALIDATE"]]); + expect(types(msgs)).toEqual(["DIRTY", "DATA", "INVALIDATE"]); + expect(s.cache).toBeUndefined(); + }); + + it("INVALIDATE + INVALIDATE collapse to one (Q9)", () => { + const s = node([], null, { initial: 1 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["INVALIDATE"], ["INVALIDATE"]]); + expect(types(msgs)).toEqual(["INVALIDATE"]); + }); +}); + +describe("TEARDOWN (R-teardown-complete)", () => { + it("non-terminal TEARDOWN synthesizes a COMPLETE prefix", () => { + const s = node([], null, { initial: 1 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["TEARDOWN"]]); + expect(types(msgs)).toEqual(["COMPLETE", "TEARDOWN"]); + }); + + it("does not stack COMPLETE when the wave already has one", () => { + const s = node([], null, { initial: 1 }); + const { msgs } = collect(s); + msgs.length = 0; + s.down([["COMPLETE"], ["TEARDOWN"]]); + expect(types(msgs)).toEqual(["COMPLETE", "TEARDOWN"]); + }); +}); + +describe("ROM/RAM + cleanup (R-rom-ram, R-cleanup-hooks)", () => { + it("compute node clears cache on deactivation and fires onDeactivation", () => { + const a = node([], null, { initial: 1 }); + let cleaned = 0; + const d = node([a], (ctx: Ctx) => { + ctx.onDeactivation(() => cleaned++); + ctx.down([["DATA", 1]]); + }); + const { unsub } = collect(d); + expect(d.cache).toBe(1); + unsub(); + expect(cleaned).toBe(1); + expect(d.cache).toBeUndefined(); // RAM + expect(d.status).toBe("sentinel"); + }); + + it("state node retains cache across disconnect (ROM)", () => { + const s = node([], null, { initial: 7 }); + const { unsub } = collect(s); + unsub(); + expect(s.cache).toBe(7); + }); +}); diff --git a/packages/ts/src/__tests__/qa-fixes.test.ts b/packages/ts/src/__tests__/qa-fixes.test.ts new file mode 100644 index 00000000..c6a54f20 --- /dev/null +++ b/packages/ts/src/__tests__/qa-fixes.test.ts @@ -0,0 +1,63 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { node } from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const types = (msgs: Message[]) => msgs.map((m) => m[0]); + +describe("QA fixes", () => { + it("EC1: rejects bare/undefined DATA payload (R-data-payload)", () => { + const s = node([], null, { initial: 1 }); + collect(s); + expect(() => s.down([["DATA", undefined as unknown as number]])).toThrow(/non-SENTINEL/); + }); + + it("EC2: rejects DATA + RESOLVED in one wave (R-equals tier-3 exclusivity)", () => { + const s = node([], null, { initial: 1 }); + collect(s); + expect(() => s.down([["DATA", 2], ["RESOLVED"]])).toThrow(/tier-3 exclusivity/); + }); + + it("EC3: dep INVALIDATE after DIRTY (before any DATA) un-wedges downstream", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null); // uncached -> first-run gate holds d forever + const d = node([a, b], (ctx: Ctx) => ctx.down([["DATA", 1]])); + const { msgs } = collect(d); + expect(d.cache).toBeUndefined(); // gate held (b never settled) + msgs.length = 0; + + a.down([["DIRTY"]]); // d goes dirty, broadcasts DIRTY (pending=1) + expect(types(msgs)).toEqual(["DIRTY"]); + + a.down([["INVALIDATE"]]); // a had data -> emits INVALIDATE to d -> d un-wedges + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); // downstream un-dirtied, not stuck + expect(d.status).toBe("sentinel"); + }); + + it("BH3: terminal while paused discards the buffer (no post-terminal DATA on resume)", () => { + const s = node([], null, { initial: 0, pausable: "resumeAll" }); + const { msgs } = collect(s); + msgs.length = 0; + const L = Symbol("L"); + s.up([["PAUSE", L]]); + s.down([["DATA", 1]]); // buffered (resumeAll) + s.down([["COMPLETE"]]); // terminal bypasses buffer + discards it + expect(types(msgs)).toContain("COMPLETE"); + s.up([["RESUME", L]]); // resume -> terminal guard -> no drain + expect(types(msgs)).not.toContain("DATA"); // buffered DATA never delivered post-terminal + }); + + it("BH6: INVALIDATE clears the replay buffer (no stale replay)", () => { + const s = node([], null, { replayBuffer: 3 }); + collect(s); + s.down([["DATA", 1]]); + s.down([["DATA", 2]]); + s.down([["INVALIDATE"]]); + const { msgs } = collect(s); // late subscriber + expect(types(msgs)).toEqual(["START"]); // no stale DATA replayed + }); +}); diff --git a/packages/ts/src/batch/batch.ts b/packages/ts/src/batch/batch.ts new file mode 100644 index 00000000..c8bbe7ba --- /dev/null +++ b/packages/ts/src/batch/batch.ts @@ -0,0 +1,92 @@ +/** + * Declarative batch (R-batch-coalesce / D12). + * + * Inside a batch, DIRTY propagates immediately but the tier-3 settle slice + * (DATA/RESOLVED/INVALIDATE) is deferred to commit, so a shared downstream + * recomputes ONCE after all batched sources settle. Success -> commit; a thrown + * error -> rollback; `bctx.rollback()` is the explicit escape hatch. + * + * Coalescing note: this kernel coalesces multiple emits to the SAME node within + * one batch to the latest tier-3 wave (last-value-wins). Full per-item batch-array + * delivery (downstream fn receiving [v1..vK]) is a documented refinement — the + * dominant use (coalescing a diamond join to one recompute) is exact here. + */ + +import type { Wave } from "../protocol/messages.js"; + +/** A node target the batch can commit/rollback against (structural, avoids an import cycle). */ +export interface BatchTarget { + __commitBatchedWave(wave: Wave): void; + __rollbackBatched(): void; +} + +export interface BatchCtx { + /** Discard all deferred emissions in this batch instead of committing. */ + rollback(): void; +} + +interface ActiveBatch { + order: BatchTarget[]; + deferred: Map; + rolledBack: boolean; +} + +let active: ActiveBatch | null = null; + +export function currentBatch(): boolean { + return active !== null; +} + +/** + * Defer a node's tier-3 settle slice into the active batch. Returns true if a batch + * captured it; false if there is no active batch (caller emits normally). + */ +export function deferToBatch(target: BatchTarget, tier3Wave: Wave): boolean { + if (active === null) return false; + if (!active.deferred.has(target)) active.order.push(target); + active.deferred.set(target, tier3Wave); // last-value coalescing + return true; +} + +function commit(b: ActiveBatch): void { + for (const target of b.order) { + const wave = b.deferred.get(target); + if (wave) target.__commitBatchedWave(wave); + } +} + +function rollback(b: ActiveBatch): void { + for (const target of b.order) target.__rollbackBatched(); +} + +/** Run `fn` as a batch (D12). DATA deferred to commit; throw/rollback discards. */ +export function batch(fn: (bctx: BatchCtx) => R): R { + if (active !== null) { + // Nested batch joins the outer frame (one commit at the outermost exit). + const outer = active; + return fn({ + rollback: () => { + outer.rolledBack = true; + }, + }); + } + const b: ActiveBatch = { order: [], deferred: new Map(), rolledBack: false }; + active = b; + const bctx: BatchCtx = { + rollback: () => { + b.rolledBack = true; + }, + }; + let result: R; + try { + result = fn(bctx); + } catch (e) { + active = null; + rollback(b); + throw e; + } + active = null; + if (b.rolledBack) rollback(b); + else commit(b); + return result; +} diff --git a/packages/ts/src/ctx/types.ts b/packages/ts/src/ctx/types.ts new file mode 100644 index 00000000..85b4d0b9 --- /dev/null +++ b/packages/ts/src/ctx/types.ts @@ -0,0 +1,63 @@ +/** + * The fn-invocation context and per-dep record shapes. + * + * Canonical authority: ~/src/graphrefly/spec/rules.jsonl + * R-fn-contract (D8/D27), R-ctx-state (D23/D29), R-cleanup-hooks (D28), R-ctx-up. + */ + +import type { Message, Wave } from "../protocol/messages.js"; + +/** A downstream sink callback (the only way to connect to a node's output). */ +export type Sink = (msg: Message) => void; + +/** + * Per-dependency record visible to a node's fn (R-fn-contract). One per declared dep. + */ +export interface DepRecord { + /** DATA values this dep delivered in the current wave; `null` = dep not involved, `[]` = RESOLVED-only. */ + batch: readonly T[] | null; + /** Last DATA value from any wave (incl prior); SENTINEL (`undefined`) = never emitted DATA. */ + prevData: T | undefined; + /** Convenience: latest DATA = last of `batch` if present, else `prevData`. */ + latest: T | undefined; + /** Tier of this dep's most recent message in the current wave (0 if none). */ + tier: number; + /** Terminal state: `undefined` = live, `true` = COMPLETE, otherwise the ERROR payload. */ + terminal: true | unknown | undefined; +} + +/** + * Per-node private cross-wave state (R-ctx-state / D23,D29). Default fresh-lifecycle + * wipe; `persist(true)` keeps it across lifecycle transitions. + */ +export interface CtxState { + get(): S | undefined; + set(v: S): void; + persist(on?: boolean): void; +} + +/** + * The single argument to a node fn: `(ctx) => void` (R-fn-contract / D8). All emission + * is explicit via `ctx.down`; there is no return-value framing. + */ +export interface Ctx { + /** Emit upstream toward deps — control tiers only (R-ctx-up). */ + up(msgs: Wave): void; + /** Emit downstream toward sinks. */ + down(msgs: Wave): void; + depRecords: readonly DepRecord[]; + state: CtxState; + /** Release external resources on deactivation (R-cleanup-hooks). */ + onDeactivation(fn: () => void): void; + /** Flush on INVALIDATE (R-cleanup-hooks). */ + onInvalidate(fn: () => void): void; + /** + * Read a dep's latest value by index (dynamicNode only, R-dynamic-node / D35). + * Present only on dynamicNode fns; all declared deps still participate in wave + * tracking, but an unread dep's change leaves the output unchanged -> equals absorbs. + */ + track?(depIndex: number): unknown; +} + +/** A node fn (R-fn-contract). Registered in a dispatcher pool, indexed by Handle. */ +export type NodeFn = (ctx: Ctx) => void; diff --git a/packages/ts/src/dispatcher/index.ts b/packages/ts/src/dispatcher/index.ts new file mode 100644 index 00000000..21c6ef5c --- /dev/null +++ b/packages/ts/src/dispatcher/index.ts @@ -0,0 +1,103 @@ +/** + * Dispatcher, pools, and the pure-data Handle. + * + * Canonical authority: ~/src/graphrefly/spec/rules.jsonl + * R-handle (D7), R-dispatch-all (D21), R-sync-core (D20/D21). + * + * Lifted from the validated handle-dispatch PoC (packages/pure-ts r8-poc, R8/R9): + * fn lives in an external pool, indexed by handle; `dispatcher.invoke` is uniformly + * SYNC void — async/remote behavior lives in the fn body (it kicks off async work + * and emits later via `ctx.down`), not in a different call mechanism. + */ + +import type { Ctx, NodeFn } from "../ctx/types.js"; + +/** A pool kind label. LocalSync + LocalAsync ship in 1.0 (D20). */ +export type PoolKind = "sync" | "async"; + +/** + * Handle = pure data `(poolId, handleId)`, NO methods (R-handle / D7). A serializable + * index into a pool's dispatch table. node != handle. + */ +export interface Handle { + readonly poolId: number; + readonly handleId: number; +} + +/** A dispatch pool: a flat fn table + an array-indexed sync invoke. */ +export interface Pool { + readonly kind: PoolKind; + register(fn: NodeFn): number; + invoke(handleId: number, ctx: Ctx): void; +} + +class LocalSyncPool implements Pool { + readonly kind = "sync" as const; + private fns: NodeFn[] = []; + register(fn: NodeFn): number { + const id = this.fns.length; + this.fns.push(fn); + return id; + } + invoke(handleId: number, ctx: Ctx): void { + this.fns[handleId](ctx); + } +} + +/** + * Structurally identical to LocalSync (R9): the "async" nature lives in the fn body, + * not the pool's invoke. The label informs ctx lifecycle (per-invocation ctx, L3-Q5) + * at the node, not a different dispatch path. + */ +class LocalAsyncPool implements Pool { + readonly kind = "async" as const; + private fns: NodeFn[] = []; + register(fn: NodeFn): number { + const id = this.fns.length; + this.fns.push(fn); + return id; + } + invoke(handleId: number, ctx: Ctx): void { + this.fns[handleId](ctx); + } +} + +/** + * First-class dispatcher (D21). Owns pools; graph binds to one (default = process-global, + * D26 — the only global singleton). Pool trait is pluggable for WorkerPool/RemotePool (D20). + */ +export class Dispatcher { + private pools: Pool[] = []; + readonly syncPoolId: number; + readonly asyncPoolId: number; + + constructor() { + this.syncPoolId = this.addPool(new LocalSyncPool()); + this.asyncPoolId = this.addPool(new LocalAsyncPool()); + } + + addPool(pool: Pool): number { + const id = this.pools.length; + this.pools.push(pool); + return id; + } + + /** Register a fn in a pool, returning its Handle. Default pool = sync (R-sync-core). */ + register(fn: NodeFn, pool: PoolKind | number = "sync"): Handle { + const poolId = pool === "sync" ? this.syncPoolId : pool === "async" ? this.asyncPoolId : pool; + const handleId = this.pools[poolId].register(fn); + return { poolId, handleId }; + } + + /** Uniform sync-void invoke (R-sync-core / R-dispatch-all). */ + invoke(handle: Handle, ctx: Ctx): void { + this.pools[handle.poolId].invoke(handle.handleId, ctx); + } + + poolKind(poolId: number): PoolKind { + return this.pools[poolId].kind; + } +} + +/** The only global singleton (D26). Overridable by passing an explicit dispatcher. */ +export const defaultDispatcher = new Dispatcher(); diff --git a/packages/ts/src/index.ts b/packages/ts/src/index.ts new file mode 100644 index 00000000..7f59bf53 --- /dev/null +++ b/packages/ts/src/index.ts @@ -0,0 +1,19 @@ +/** + * @graphrefly/ts — clean-slate TypeScript substrate (CSP-1 kernel). + * + * Canonical authority: ~/src/graphrefly/spec/rules.jsonl + decisions.jsonl (D1-D36). + * Scope: node / dispatcher / pool / wave protocol. The graph layer, 8-verb sugar, + * operators, and inspection are CSP-2. + */ + +export { type BatchCtx, batch } from "./batch/batch.js"; +export type { Ctx, CtxState, DepRecord, NodeFn, Sink } from "./ctx/types.js"; +export { + Dispatcher, + defaultDispatcher, + type Handle, + type Pool, + type PoolKind, +} from "./dispatcher/index.js"; +export { dynamicNode, Node, type NodeOptions, node, type Status } from "./node/node.js"; +export * from "./protocol/messages.js"; diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts new file mode 100644 index 00000000..43e62d3f --- /dev/null +++ b/packages/ts/src/node/node.ts @@ -0,0 +1,805 @@ +/** + * The node primitive — the thinnest substrate object (R-node-thin / D5). + * + * Holds a fn handle + deps + the wave state machine; ZERO inspection cruft + * (naming/find/describe are the graph layer, CSP-2). Canonical authority: + * ~/src/graphrefly/spec/rules.jsonl (R-node-*, R-two-phase, R-diamond, R-equals, + * R-first-run-gate, R-push-subscribe, R-rom-ram, R-fn-contract, R-initial, R-ctx-up). + * + * Slice 1 = core wave: state node, compute node, two-phase DIRTY->DATA, diamond + * pending-counter join, first-run gate, equals->RESOLVED, push-on-subscribe, + * lazy activation, ROM/RAM. Lifecycle (terminal/INVALIDATE/cleanup), control + * (PAUSE/async), batch, and dynamicNode land in later slices. + */ + +import { currentBatch, deferToBatch } from "../batch/batch.js"; +import type { Ctx, CtxState, DepRecord, NodeFn, Sink } from "../ctx/types.js"; +import { type Dispatcher, defaultDispatcher, type Handle } from "../dispatcher/index.js"; +import { + isUpAllowed, + type Message, + messageTier, + SENTINEL, + type Wave, +} from "../protocol/messages.js"; + +export type Status = + | "sentinel" + | "pending" + | "dirty" + | "settled" + | "resolved" + | "completed" + | "errored"; + +export interface NodeOptions { + /** Pre-populate cache; source pushes [DATA, initial] on subscribe (R-initial). `null` is valid. */ + initial?: T | null; + /** Custom equality for the DATA->RESOLVED substitution (R-equals). Default Object.is. */ + equals?: (a: T, b: T) => boolean; + /** First-run gate off when true; fn body must guard SENTINEL per dep (R-first-run-gate). */ + partial?: boolean; + /** A dep terminal also settles the first-run gate (Reduce-class, R-first-run-gate). */ + terminalAsRealInput?: boolean; + /** Auto-emit COMPLETE when ALL deps complete (R-deps-terminal). Default true. */ + completeWhenDepsComplete?: boolean; + /** Auto-emit ERROR when any dep errors (R-deps-terminal). Default true. */ + errorWhenDepsError?: boolean; + /** Allow re-activation after terminal; late subscribe resets the lifecycle (R-terminal). Default false. */ + resubscribable?: boolean; + /** Clear cached value on TEARDOWN. Default false. */ + resetOnTeardown?: boolean; + /** PAUSE/RESUME behavior (R-pause-modes). Default true. */ + pausable?: boolean | "resumeAll"; + /** Buffer the last N outgoing DATA for late subscribers (R-replay-buffer). */ + replayBuffer?: number; + /** Mark this as a dynamicNode — fn gets ctx.track(i) for read-selection (R-dynamic-node / D35). */ + dynamic?: boolean; + /** Dispatch pool for the fn (R-sync-core). Default sync. */ + pool?: "sync" | "async"; + /** Dispatcher to register/invoke against. Default = process-global (D26). */ + dispatcher?: Dispatcher; + /** Optional debug name (graph layer owns real naming/inspection). */ + name?: string; +} + +const defaultEquals = Object.is as (a: unknown, b: unknown) => boolean; + +export class Node { + private readonly _deps: Node[]; + private readonly _handle: Handle | null; + private readonly _dispatcher: Dispatcher; + private readonly _equals: (a: T, b: T) => boolean; + private readonly _partial: boolean; + private readonly _terminalAsRealInput: boolean; + private readonly _completeWhenDepsComplete: boolean; + private readonly _errorWhenDepsError: boolean; + private readonly _resubscribable: boolean; + private readonly _resetOnTeardown: boolean; + private readonly _pausable: boolean | "resumeAll"; + private readonly _replayN: number; + private readonly _dynamic: boolean; + readonly name?: string; + + private _subscribers = new Set(); + private _activated = false; + private _depUnsubs: Array<() => void> = []; + + // per-dep wave state + private _depBatch: Array; + private _depPrev: unknown[]; + private _depHasData: boolean[]; + private _depDirty: boolean[]; + private _depTier: number[]; + private _depTerminal: Array; + private _pending = 0; + + private _cache: T | undefined = SENTINEL; + private _hasData = false; + private _status: Status = "sentinel"; + private _hasCalledFnOnce = false; + private _emittedDirtyThisWave = false; + private _insideRunWave = false; + /** Node's own terminal: undefined = live, true = COMPLETE, else ERROR payload. */ + private _terminal: true | unknown | undefined = undefined; + private _hasTorndown = false; + + // control plane (R-pause-lockset, R-pause-modes, R-async-paused, R-replay-buffer) + private _pauseLockset = new Set(); + private _pausedDepWaveOccurred = false; + private _pauseBuffer: Wave[] = []; + private _replayRing: T[] = []; + // BH1: a DIRTY broadcast during a batch defer is owed a balancing RESOLVED on + // rollback — tracked independently of _emittedDirtyThisWave (which a fn wave resets). + private _batchDirtyOwed = false; + + // ctx.state (R-ctx-state) + private _state: unknown = SENTINEL; + private _statePersist = false; + + // cleanup hooks (R-cleanup-hooks) + private _onDeactivation: Array<() => void> = []; + private _onInvalidate: Array<() => void> = []; + + // reusable depRecords + sync ctx (L3-Q5 node-stable ctx for the sync pool) + private _depRecords: DepRecord[]; + private _syncCtx: Ctx | null = null; + + constructor( + deps: Node[], + handleOrFn: Handle | NodeFn | null, + opts: NodeOptions = {}, + ) { + this._deps = deps; + this._dispatcher = opts.dispatcher ?? defaultDispatcher; + this._equals = (opts.equals ?? defaultEquals) as (a: T, b: T) => boolean; + this._partial = opts.partial ?? false; + this._terminalAsRealInput = opts.terminalAsRealInput ?? false; + this._completeWhenDepsComplete = opts.completeWhenDepsComplete ?? true; + this._errorWhenDepsError = opts.errorWhenDepsError ?? true; + this._resubscribable = opts.resubscribable ?? false; + this._resetOnTeardown = opts.resetOnTeardown ?? false; + this._pausable = opts.pausable ?? true; + this._replayN = opts.replayBuffer ?? 0; + this._dynamic = opts.dynamic ?? false; + this.name = opts.name; + + if (handleOrFn === null) this._handle = null; + else if (typeof handleOrFn === "function") + this._handle = this._dispatcher.register(handleOrFn, opts.pool ?? "sync"); + else this._handle = handleOrFn; + + const n = deps.length; + this._depBatch = new Array(n).fill(null); + this._depPrev = new Array(n).fill(SENTINEL); + this._depHasData = new Array(n).fill(false); + this._depDirty = new Array(n).fill(false); + this._depTier = new Array(n).fill(0); + this._depTerminal = new Array(n).fill(undefined); + this._depRecords = deps.map(() => ({ + batch: null, + prevData: SENTINEL, + latest: SENTINEL, + tier: 0, + terminal: undefined, + })); + + // R-initial: a provided initial (incl null) pre-populates the cache. + if (opts.initial !== undefined) { + this._cache = opts.initial as T; + this._hasData = true; + this._status = "settled"; + } + } + + get cache(): T | undefined { + return this._cache; + } + + get status(): Status { + return this._status; + } + + /** R-push-subscribe: a new sink receives START, then cached DATA (or DIRTY if dirty). */ + subscribe(sink: Sink): () => void { + // R-terminal: late subscribe to a terminal node either resets (resubscribable) + // or is rejected (non-resubscribable, R2.2.7.b). + if (this._terminal !== undefined) { + if (this._resubscribable) this._resetLifecycle(); + else + throw new Error( + "subscribe: node is non-resubscribable and has terminated; the stream is permanently over (R-terminal / R2.2.7.b)", + ); + } + + this._subscribers.add(sink); + sink(["START"]); + if (this._replayN > 0 && this._replayRing.length > 0) { + // R-replay-buffer: late subscriber gets the last N DATA after START. + for (const v of this._replayRing) sink(["DATA", v]); + } else if (this._hasData) { + sink(["DATA", this._cache]); + } else if (this._status === "dirty") { + sink(["DIRTY"]); + } + + if (!this._activated) this._activate(); + + return () => { + if (!this._subscribers.delete(sink)) return; + if (this._subscribers.size === 0) this._deactivate(); + }; + } + + /** External emission toward sinks (state-node push, or async late-emit). One call = one wave. */ + down(msgs: Wave): void { + this._down(msgs); + } + + /** Emit upstream toward deps — control tiers only (R-ctx-up). */ + up(msgs: Wave): void { + this._up(msgs); + } + + // ── activation / deactivation (lazy; R-rom-ram) ── + + private _activate(): void { + this._activated = true; + for (let i = 0; i < this._deps.length; i++) { + const idx = i; + const unsub = this._deps[i].subscribe((msg) => this._receiveFromDep(idx, msg)); + this._depUnsubs.push(unsub); + } + // Depless producer (fn, no deps): run once on activation. + if (this._deps.length === 0 && this._handle !== null && !this._hasCalledFnOnce) { + this._runWave(); + } + } + + private _deactivate(): void { + this._activated = false; + for (const u of this._depUnsubs) u(); + this._depUnsubs = []; + for (const fn of this._onDeactivation) fn(); + this._onDeactivation = []; + this._onInvalidate = []; + + const isCompute = this._handle !== null || this._deps.length > 0; + if (isCompute) { + // RAM: compute nodes clear cache; reconnect re-runs fn fresh. + this._cache = SENTINEL; + this._hasData = false; + this._status = "sentinel"; + } + this._resetDepState(); + this._hasCalledFnOnce = false; + this._pauseLockset.clear(); + this._pauseBuffer = []; + this._pausedDepWaveOccurred = false; + this._replayRing = []; // BH6: don't replay stale values to a post-reactivation subscriber + if (!this._statePersist) this._state = SENTINEL; + } + + private _resetDepState(): void { + const n = this._deps.length; + for (let i = 0; i < n; i++) { + this._depBatch[i] = null; + this._depPrev[i] = SENTINEL; + this._depHasData[i] = false; + this._depDirty[i] = false; + this._depTier[i] = 0; + this._depTerminal[i] = undefined; + } + this._pending = 0; + this._emittedDirtyThisWave = false; + } + + // ── upstream wave receive (two-phase + diamond) ── + + private _receiveFromDep(idx: number, msg: Message): void { + const t = msg[0]; + if (t === "START") return; + // Terminal-is-forever: a terminated node ignores further upstream messages. + if (this._terminal !== undefined) return; + + if (t === "INVALIDATE") { + // The dep's value is gone — drop our view of it (prevData -> SENTINEL so the + // never-emitted detector reads correctly, C-3) and cascade (idempotent). + this._depPrev[idx] = SENTINEL; + this._depHasData[idx] = false; + this._depBatch[idx] = null; + // EC3: un-wedge the dirty bookkeeping if this dep had gone DIRTY first, so an + // INVALIDATE-before-DATA doesn't strand _pending / downstream forever + // (R-invalidate-idempotent — exists to prevent the wedged-DIRTY deadlock). + if (this._depDirty[idx]) { + this._depDirty[idx] = false; + this._pending--; + } + const hadData = this._hasData; + this._invalidate(); // cascades INVALIDATE iff populated; no-op otherwise + // If we broadcast DIRTY this wave but _invalidate produced no settle (the node + // was never populated, so the cascade is suppressed per the rule), un-dirty + // downstream with a RESOLVED once all deps have settled. + if (this._pending === 0 && this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = false; + if (!hadData) { + this._status = "sentinel"; + this._emitToSubs(["RESOLVED"]); + } + } + return; + } + + if (t === "COMPLETE") { + this._depTerminal[idx] = true; + if (this._completeWhenDepsComplete && this._allDepsComplete()) { + this._down([["COMPLETE"]]); + } else if (this._terminalAsRealInput) { + this._maybeRun(); + } + return; + } + + if (t === "ERROR") { + this._depTerminal[idx] = msg[1]; + if (this._errorWhenDepsError) { + this._down([["ERROR", msg[1]]]); + } else if (this._terminalAsRealInput) { + this._maybeRun(); + } + return; + } + + if (t === "TEARDOWN") { + this._down([["TEARDOWN"]]); + return; + } + + if (t === "DIRTY") { + if (!this._depDirty[idx]) { + this._depDirty[idx] = true; + this._pending++; + this._depTier[idx] = 2; + this._markDirty(); + } + return; + } + + if (t === "DATA") { + const v = msg[1]; + const b = this._depBatch[idx]; + if (b === null) this._depBatch[idx] = [v]; + else b.push(v); + this._depPrev[idx] = v; + this._depHasData[idx] = true; + this._depTier[idx] = 3; + if (this._depDirty[idx]) { + this._depDirty[idx] = false; + this._pending--; + } + this._maybeRun(); + return; + } + + if (t === "RESOLVED") { + this._depTier[idx] = 3; + if (this._depDirty[idx]) { + this._depDirty[idx] = false; + this._pending--; + } + this._maybeRun(); + return; + } + // PAUSE / RESUME are not delivered downstream to a dep-subscriber; a node is + // paused via its own up() (lockset), not by an upstream dep. + } + + private _markDirty(): void { + this._status = "dirty"; + if (!this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = true; + this._emitToSubs(["DIRTY"]); + } + } + + private _maybeRun(): void { + // R-pause-modes (default): while paused, skip dep-driven fn re-execution and + // coalesce — fire once with the latest dep values on final-lock RESUME. + if (this._pausable === true && this._isPaused()) { + this._pausedDepWaveOccurred = true; + return; + } + this._tryRun(); + } + + private _tryRun(): void { + if (this._pending > 0) return; + if (this._handle === null) { + // Passthrough wire (deps, no fn): forward the latest dep DATA downstream. + this._passthroughEmit(); + return; + } + if (!this._hasCalledFnOnce) { + if (this._partial || this._allDepsSettled()) this._runWave(); + // else: first-run gate holds fn until every dep has settled (R-first-run-gate). + return; + } + this._runWave(); + } + + private _allDepsSettled(): boolean { + for (let i = 0; i < this._deps.length; i++) { + if (this._depHasData[i]) continue; + if (this._terminalAsRealInput && this._depTerminal[i] !== undefined) continue; + return false; + } + return true; + } + + private _passthroughEmit(): void { + // Single-dep wire: relay dep 0's latest batch value as DATA. + const b = this._depBatch[0]; + if (b !== null && b.length > 0) { + this._down([["DATA", b[b.length - 1]]]); + } + this._depBatch[0] = null; + this._emittedDirtyThisWave = false; + } + + private _runWave(): void { + this._hasCalledFnOnce = true; + const ctx = this._buildCtx(); + this._insideRunWave = true; + this._dispatcher.invoke(this._handle as Handle, ctx); + this._insideRunWave = false; + + // roll wave-local state forward + for (let i = 0; i < this._depBatch.length; i++) this._depBatch[i] = null; + this._emittedDirtyThisWave = false; + } + + // ── ctx construction (L3-Q5: sync = node-stable reused ctx; async = per-invocation) ── + + private _buildCtx(): Ctx { + for (let i = 0; i < this._deps.length; i++) { + const batch = this._depBatch[i]; + const prev = this._depPrev[i]; + const rec = this._depRecords[i]; + rec.batch = batch as readonly unknown[] | null; + rec.prevData = prev; + rec.latest = batch && batch.length > 0 ? batch[batch.length - 1] : prev; + rec.tier = this._depTier[i]; + rec.terminal = this._depTerminal[i]; + } + + const kind = this._handle ? this._dispatcher.poolKind(this._handle.poolId) : "sync"; + if (kind === "sync") { + if (this._syncCtx === null) this._syncCtx = this._makeCtx(this._depRecords); + return this._syncCtx; + } + // async: snapshot depRecords so a deferred late-emit reads this wave's view. + return this._makeCtx(this._depRecords.map((r) => ({ ...r }))); + } + + private _makeCtx(depRecords: readonly DepRecord[]): Ctx { + const ctx: Ctx = { + up: (msgs) => this._up(msgs), + down: (msgs) => this._down(msgs), + depRecords, + state: this._makeState(), + onDeactivation: (fn) => { + this._onDeactivation.push(fn); + }, + onInvalidate: (fn) => { + this._onInvalidate.push(fn); + }, + }; + if (this._dynamic) { + // R-dynamic-node: read a dep's latest by index. Untracked deps still drive + // waves; if the output is unchanged, equals absorbs them (RESOLVED). + ctx.track = (i: number) => ctx.depRecords[i]?.latest; + } + return ctx; + } + + private _makeState(): CtxState { + return { + get: () => this._state as S | undefined, + set: (v: S) => { + this._state = v; + }, + persist: (on = true) => { + this._statePersist = on; + }, + }; + } + + // ── downstream emission pipeline (the unified waist) ── + + private _down(msgs: Wave): void { + let sorted: Message[] = [...msgs].sort((a, b) => messageTier(a[0]) - messageTier(b[0])); + // R-same-wave-merge: collapse repeated INVALIDATE in one wave (Q9) so the + // cleanup hook + downstream broadcast fire at most once. + const firstInvalidate = sorted.findIndex((m) => m[0] === "INVALIDATE"); + if (firstInvalidate !== -1) { + sorted = sorted.filter((m, i) => m[0] !== "INVALIDATE" || i === firstInvalidate); + } + // R-teardown-complete: a TEARDOWN reaching a non-terminal node synthesizes a + // COMPLETE prefix (so firstValueFrom-style bridges resolve), unless the wave + // already carries a terminal or the node already tore down. + const hasTeardown = sorted.some((m) => m[0] === "TEARDOWN"); + const hasTerminal = sorted.some((m) => m[0] === "COMPLETE" || m[0] === "ERROR"); + if (hasTeardown && !hasTerminal && this._terminal === undefined && !this._hasTorndown) { + sorted = [["COMPLETE"], ...sorted]; + } + // R-batch-coalesce (D12): inside a batch, emit DIRTY immediately but defer the + // tier-3 settle slice to commit so a shared downstream recomputes once. Only + // external emits defer (fn emits during commit run normally). + if (!this._insideRunWave && currentBatch()) { + const tier3 = sorted.filter((m) => messageTier(m[0]) >= 3); + if (tier3.length > 0) { + if (!this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = true; + this._status = "dirty"; + this._emitToSubs(["DIRTY"]); + } + this._batchDirtyOwed = true; // BH1: owe a balancing RESOLVED on rollback + deferToBatch(this, tier3); + return; + } + } + // R-pause-modes / R-async-paused: defer the settle slice (tier 3/4) into the + // pause buffer while paused; tier 0-2 (DIRTY/PAUSE/RESUME), tier 5 (terminal), + // tier 6 (TEARDOWN) bypass so end-of-stream + control always reach observers. + if (this._shouldBufferOnPause()) { + const buffered = sorted.filter((m) => { + const t = messageTier(m[0]); + return t === 3 || t === 4; + }); + if (buffered.length > 0) this._pauseBuffer.push(buffered); + sorted = sorted.filter((m) => { + const t = messageTier(m[0]); + return t !== 3 && t !== 4; + }); + if (sorted.length === 0) return; + } + let dataCount = 0; + let hasTier3 = false; + let hasResolved = false; + for (const m of sorted) { + if (m[0] === "DATA") dataCount++; + if (m[0] === "RESOLVED") hasResolved = true; + if (messageTier(m[0]) === 3) hasTier3 = true; + } + // EC2 / R-equals tier-3 exclusivity: a wave's tier-3 slot is >=1 DATA XOR exactly + // 1 RESOLVED — never mixed. Reject the protocol violation fail-fast. + if (dataCount >= 1 && hasResolved) { + throw new Error("down: a wave cannot mix DATA and RESOLVED (tier-3 exclusivity, R-equals)"); + } + + // Synthesize a leading DIRTY for an EXTERNAL tier-3 emit (R-dirty-before-data). + // Inside runWave the DIRTY was already propagated (or the wave is activation-exempt). + if (hasTier3 && !this._insideRunWave && !this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = true; + this._status = "dirty"; + this._emitToSubs(["DIRTY"]); + } + + for (const m of sorted) { + if (m[0] === "DIRTY") { + if (!this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = true; + this._status = "dirty"; + this._emitToSubs(["DIRTY"]); + } + continue; + } + if (m[0] === "DATA") { + if (m[1] === undefined) { + // EC1 / R-data-payload: bare [DATA] / [DATA, SENTINEL] is rejected. + throw new Error("down: DATA requires a non-SENTINEL payload (R-data-payload)"); + } + const v = m[1] as T; + // R-equals: DATA->RESOLVED substitution ONLY on a single-DATA wave. + if (dataCount === 1 && this._hasData && this._equals(this._cache as T, v)) { + this._status = "resolved"; + this._emitToSubs(["RESOLVED"]); + } else { + this._cache = v; + this._hasData = true; + this._status = "settled"; + if (this._replayN > 0) { + this._replayRing.push(v); + if (this._replayRing.length > this._replayN) this._replayRing.shift(); + } + this._emitToSubs(["DATA", v]); + } + continue; + } + if (m[0] === "RESOLVED") { + this._status = "resolved"; + this._emitToSubs(["RESOLVED"]); + continue; + } + if (m[0] === "INVALIDATE") { + this._invalidate(); + continue; + } + if (m[0] === "COMPLETE") { + if (this._terminal !== undefined) continue; + this._terminal = true; + this._pauseBuffer = []; // BH3: terminal discards buffered settle slices + this._status = "completed"; + this._emitToSubs(["COMPLETE"]); + continue; + } + if (m[0] === "ERROR") { + if (this._terminal !== undefined) continue; + if (m[1] === undefined) { + throw new Error("down: ERROR requires a non-SENTINEL payload (R-data-payload)"); + } + this._terminal = m[1]; + this._pauseBuffer = []; // BH3: terminal discards buffered settle slices + this._status = "errored"; + this._emitToSubs(["ERROR", m[1]]); + continue; + } + if (m[0] === "TEARDOWN") { + this._hasTorndown = true; + if (this._resetOnTeardown) { + this._cache = SENTINEL; + this._hasData = false; + } + this._emitToSubs(["TEARDOWN"]); + } + // PAUSE / RESUME — control slice. + } + + if (!this._insideRunWave) this._emittedDirtyThisWave = false; + } + + private _up(msgs: Wave): void { + for (const m of msgs) { + if (!isUpAllowed(m[0])) { + throw new Error( + `ctx.up: ${m[0]} is down-only (tier ${messageTier(m[0])}); up carries control tiers only (R-ctx-up)`, + ); + } + } + for (const m of msgs) { + if (m[0] === "PAUSE") { + this._pauseAcquire(m[1]); + } else if (m[0] === "RESUME") { + this._pauseRelease(m[1]); + } else { + // DIRTY / INVALIDATE / TEARDOWN forward upstream toward deps (observers react). + for (const dep of this._deps) dep.up([m]); + } + } + } + + // ── PAUSE/RESUME lockset (R-pause-lockset) + modes (R-pause-modes) ── + + private _isPaused(): boolean { + return this._pauseLockset.size > 0; + } + + private _isAsyncPool(): boolean { + return this._handle !== null && this._dispatcher.poolKind(this._handle.poolId) === "async"; + } + + private _pauseAcquire(lockId: unknown): void { + this._pauseLockset.add(lockId); // Set => same-id repeat PAUSE is idempotent + } + + private _pauseRelease(lockId: unknown): void { + if (!this._pauseLockset.has(lockId)) return; // unknown id => no-op + this._pauseLockset.delete(lockId); + if (this._pauseLockset.size > 0) return; // another lock still held => stay paused + this._onResume(); + } + + private _onResume(): void { + // BH3: a node that terminated while paused discards its buffer and never + // replays/recomputes (terminal-is-forever). + if (this._terminal !== undefined) { + this._pauseBuffer = []; + this._pausedDepWaveOccurred = false; + return; + } + // Drain buffered settle slices (resumeAll / async-at-paused, R-async-paused). + if (this._pauseBuffer.length > 0) { + const buf = this._pauseBuffer; + this._pauseBuffer = []; + for (const wave of buf) this._down(wave); + } + // Default mode: a dep wave that arrived while paused fires the fn once now. + if (this._pausedDepWaveOccurred) { + this._pausedDepWaveOccurred = false; + this._tryRun(); + } + } + + /** Should an outgoing settle slice be deferred into the pause buffer? */ + private _shouldBufferOnPause(): boolean { + if (!this._isPaused()) return false; + if (this._pausable === "resumeAll") return true; + // R-async-paused / DR-3: a late emit (outside runWave) from an async-pool node buffers. + if (!this._insideRunWave && this._isAsyncPool()) return true; + return false; + } + + /** R-invalidate-idempotent: clear cache + flush + cascade; no-op if nothing cached. */ + private _invalidate(): void { + if (!this._hasData) return; // never-populated or already-reset → no-op + this._cache = SENTINEL; + this._hasData = false; + this._status = "sentinel"; + this._replayRing = []; // BH6: invalidated values are stale — don't replay them + for (const fn of this._onInvalidate) fn(); + this._emitToSubs(["INVALIDATE"]); + } + + private _allDepsComplete(): boolean { + if (this._deps.length === 0) return false; + for (const tm of this._depTerminal) if (tm !== true) return false; + return true; + } + + /** R-terminal: resubscribable reset clears terminal + dep state + re-arms the gate. */ + private _resetLifecycle(): void { + for (const u of this._depUnsubs) u(); + this._depUnsubs = []; + this._activated = false; + this._terminal = undefined; + this._hasTorndown = false; + this._hasCalledFnOnce = false; + this._resetDepState(); + this._pauseLockset.clear(); + this._pauseBuffer = []; + this._pausedDepWaveOccurred = false; + this._replayRing = []; // BH6 + const isCompute = this._handle !== null || this._deps.length > 0; + if (isCompute) { + this._cache = SENTINEL; + this._hasData = false; + this._status = "sentinel"; + } else { + this._status = this._hasData ? "settled" : "sentinel"; + } + if (!this._statePersist) this._state = SENTINEL; + } + + private _emitToSubs(msg: Message): void { + // Copy guards against subscribe/unsubscribe during iteration. + const subs = [...this._subscribers]; + for (const sink of subs) sink(msg); + } + + /** Batch commit (R-batch-coalesce): deliver the deferred tier-3 wave now. */ + __commitBatchedWave(wave: Wave): void { + this._batchDirtyOwed = false; // commit delivers the real settle (BH1) + this._down(wave); // batch is inactive at commit -> processes normally + } + + /** Batch rollback: balance the immediate DIRTY with a RESOLVED so downstream un-dirties. */ + __rollbackBatched(): void { + // BH1: keyed on _batchDirtyOwed (not _emittedDirtyThisWave, which a fn wave between + // defer and rollback would have reset) so the balancing RESOLVED is never skipped. + if (this._batchDirtyOwed) { + this._batchDirtyOwed = false; + this._emittedDirtyThisWave = false; + this._status = this._hasData ? "settled" : "sentinel"; + this._emitToSubs(["RESOLVED"]); + } + } +} + +/** + * Construct a node (R-node-iface / D5 / L1.9 deps-first). + * node([], null, { initial }) — state node (manual source; emit via .down) + * node([], fn) — producer (runs on activation) + * node([a, b], fn) — compute / derived + * node([dep]) — passthrough wire + */ +export function node( + deps: Node[] = [], + handleOrFn: Handle | NodeFn | null = null, + opts: NodeOptions = {}, +): Node { + return new Node(deps, handleOrFn, opts); +} + +/** + * Construct a dynamicNode (R-dynamic-node / D35) — a node variant whose fn reads a + * subset of a fixed superset of deps per invocation via `ctx.track(i)`. All declared + * deps participate in wave tracking; an unread dep's change leaves the output unchanged, + * so equals absorbs it (no downstream propagation). Intra-graph only (D22). + */ +export function dynamicNode( + deps: Node[], + fn: NodeFn, + opts: NodeOptions = {}, +): Node { + return new Node(deps, fn, { ...opts, dynamic: true }); +} diff --git a/packages/ts/src/protocol/messages.ts b/packages/ts/src/protocol/messages.ts new file mode 100644 index 00000000..313ba5ff --- /dev/null +++ b/packages/ts/src/protocol/messages.ts @@ -0,0 +1,73 @@ +/** + * Wave-protocol message types and the tier const table. + * + * Canonical authority: ~/src/graphrefly/spec/rules.jsonl + * R-msg-format, R-msg-closed-set, R-tier (D34), R-data-payload, R-sentinel. + */ + +/** Opaque pause-lock identifier (R-pause-lockset). */ +export type LockId = string | symbol; + +/** + * The CLOSED set of 10 message types (R-msg-closed-set / D9 + START handshake). + * No open set, no user-defined custom types. Adding a type is a spec change. + */ +export type Message = + | readonly ["START"] + | readonly ["DIRTY"] + | readonly ["DATA", unknown] + | readonly ["RESOLVED"] + | readonly ["INVALIDATE"] + | readonly ["COMPLETE"] + | readonly ["ERROR", unknown] + | readonly ["TEARDOWN"] + | readonly ["PAUSE", LockId] + | readonly ["RESUME", LockId]; + +/** One array of messages delivered in one call = one Wave (R-wave-boundary). */ +export type Wave = readonly Message[]; + +/** The message-type tag (the first tuple element). */ +export type MessageType = Message[0]; + +/** + * SENTINEL = absence-of-DATA (R-sentinel / D16). TS representation is `undefined`. + * Never a valid DATA payload; `null` IS a valid DATA value (domain-level absence). + */ +export const SENTINEL = undefined; + +/** + * Tier const table (R-tier / D34). Tiers < 3 dispatch immediately; tiers >= 3 are + * batch-deferred. This is a compile-time const table (D18), not runtime config. + */ +const TIER: Record = { + START: 0, + PAUSE: 1, + RESUME: 1, + DIRTY: 2, + DATA: 3, + RESOLVED: 3, + INVALIDATE: 4, + COMPLETE: 5, + ERROR: 5, + TEARDOWN: 6, +}; + +export function messageTier(t: MessageType): number { + return TIER[t]; +} + +/** Tier >= 3 messages are deferred inside a batch (DATA/RESOLVED/INVALIDATE/terminal/teardown). */ +export function isDeferredTier(t: MessageType): boolean { + return TIER[t] >= 3; +} + +/** + * ctx.up carries control tiers only (R-ctx-up / DR-5): DIRTY, PAUSE, RESUME, + * INVALIDATE, TEARDOWN. DATA/RESOLVED (tier 3) and COMPLETE/ERROR (tier 5) are + * down-only. + */ +export function isUpAllowed(t: MessageType): boolean { + const tier = TIER[t]; + return tier !== 3 && tier !== 5; +} diff --git a/packages/ts/tsconfig.json b/packages/ts/tsconfig.json new file mode 100644 index 00000000..a12c2c97 --- /dev/null +++ b/packages/ts/tsconfig.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ES2022", + "moduleResolution": "bundler", + "lib": ["ES2022"], + "declaration": true, + "outDir": "dist", + "rootDir": "src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true + }, + "include": ["src"], + "exclude": ["node_modules", "dist", "**/*.test.ts"] +} diff --git a/packages/ts/tsup.config.ts b/packages/ts/tsup.config.ts new file mode 100644 index 00000000..43db9aad --- /dev/null +++ b/packages/ts/tsup.config.ts @@ -0,0 +1,9 @@ +import { defineConfig } from "tsup"; + +export default defineConfig({ + entry: ["src/index.ts"], + format: ["esm", "cjs"], + dts: true, + clean: true, + sourcemap: true, +}); diff --git a/packages/ts/vitest.config.ts b/packages/ts/vitest.config.ts new file mode 100644 index 00000000..2f671880 --- /dev/null +++ b/packages/ts/vitest.config.ts @@ -0,0 +1,7 @@ +import { defineConfig } from "vitest/config"; + +export default defineConfig({ + test: { + include: ["src/**/*.test.ts"], + }, +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 4bc3cdd1..cdfe1ad1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -630,6 +630,18 @@ importers: specifier: ^3.5.31 version: 3.5.31(typescript@5.9.3) + packages/ts: + devDependencies: + tsup: + specifier: ^8.5.1 + version: 8.5.1(postcss@8.5.8)(tsx@4.21.0)(typescript@5.9.3)(yaml@2.9.0) + typescript: + specifier: ^5.7.0 + version: 5.9.3 + vitest: + specifier: ^3.0.0 + version: 3.2.4(@types/debug@4.1.13)(@types/node@25.5.0)(jsdom@29.0.1)(lightningcss@1.32.0)(terser@5.47.1)(tsx@4.21.0)(yaml@2.9.0) + website: dependencies: '@astrojs/starlight': From 997ba4a720ec76b8b10c6be03cb06757c1ddf412 Mon Sep 17 00:00:00 2001 From: David Chen Date: Thu, 28 May 2026 20:45:31 -0700 Subject: [PATCH 003/175] refactor(ts): clean up imports and enhance test structure - Consolidated import statements in `r8-async.test.ts` for better readability. - Streamlined function signatures in `r8.ts` to improve clarity. - Reformatted `package.json` to enhance the structure of the "files" array. - Expanded the `index.ts` file with additional type exports for better type management. - Introduced new test files for comprehensive coverage of graph functionality, including conformance, graph operations, reentrancy, and upstream control. - Added detailed test cases to validate the behavior of the graph layer and its operators. --- .../__experiments__/r8-poc/r8-async.test.ts | 9 +- .../pure-ts/src/__experiments__/r8-poc/r8.ts | 15 +- packages/ts/package.json | 5 +- packages/ts/src/__tests__/conformance.test.ts | 60 ++- packages/ts/src/__tests__/graph.test.ts | 136 +++++ packages/ts/src/__tests__/inspect.test.ts | 73 +++ packages/ts/src/__tests__/operators.test.ts | 97 ++++ packages/ts/src/__tests__/reentrancy.test.ts | 43 ++ packages/ts/src/__tests__/up-source.test.ts | 78 +++ packages/ts/src/dispatcher/index.ts | 55 +- packages/ts/src/graph/describe.ts | 43 ++ packages/ts/src/graph/graph.ts | 469 ++++++++++++++++++ packages/ts/src/graph/inspect.ts | 41 ++ packages/ts/src/index.ts | 25 +- packages/ts/src/node/node.ts | 39 +- 15 files changed, 1159 insertions(+), 29 deletions(-) create mode 100644 packages/ts/src/__tests__/graph.test.ts create mode 100644 packages/ts/src/__tests__/inspect.test.ts create mode 100644 packages/ts/src/__tests__/operators.test.ts create mode 100644 packages/ts/src/__tests__/reentrancy.test.ts create mode 100644 packages/ts/src/__tests__/up-source.test.ts create mode 100644 packages/ts/src/graph/describe.ts create mode 100644 packages/ts/src/graph/graph.ts create mode 100644 packages/ts/src/graph/inspect.ts diff --git a/packages/pure-ts/src/__experiments__/r8-poc/r8-async.test.ts b/packages/pure-ts/src/__experiments__/r8-poc/r8-async.test.ts index a025b5f0..db4aaff0 100644 --- a/packages/pure-ts/src/__experiments__/r8-poc/r8-async.test.ts +++ b/packages/pure-ts/src/__experiments__/r8-poc/r8-async.test.ts @@ -9,14 +9,9 @@ */ import { describe, expect, it } from "vitest"; -import type { Actions, Ctx, TinyNode } from "./protocol.js"; import { baselineNode } from "./baseline.js"; -import { - dispatcher, - r8AsyncNode, - r8Node, - r8RemoteNode, -} from "./r8.js"; +import type { Actions, Ctx, TinyNode } from "./protocol.js"; +import { dispatcher, r8AsyncNode, r8Node, r8RemoteNode } from "./r8.js"; const lastOrPrev = (idx: number) => diff --git a/packages/pure-ts/src/__experiments__/r8-poc/r8.ts b/packages/pure-ts/src/__experiments__/r8-poc/r8.ts index 12ad2832..a67cb414 100644 --- a/packages/pure-ts/src/__experiments__/r8-poc/r8.ts +++ b/packages/pure-ts/src/__experiments__/r8-poc/r8.ts @@ -86,10 +86,7 @@ class SimulatedRemotePool { } registerRemote( - sandboxFn: ( - batchData: ReadonlyArray, - ctx: Ctx, - ) => T, + sandboxFn: (batchData: ReadonlyArray, ctx: Ctx) => T, ): number { const id = this.fns.length; const wrapper: AnyFn = (batchData, actions, ctx) => { @@ -246,10 +243,7 @@ export class R8RemoteNode extends TinyNode { constructor( deps: TinyNode[], - sandboxFn: ( - batchData: ReadonlyArray, - ctx: Ctx, - ) => T, + sandboxFn: (batchData: ReadonlyArray, ctx: Ctx) => T, ) { super(deps); const handle = dispatcher.remotePool(0).registerRemote(sandboxFn); @@ -274,10 +268,7 @@ export class R8RemoteNode extends TinyNode { export function r8RemoteNode( deps: TinyNode[], - sandboxFn: ( - batchData: ReadonlyArray, - ctx: Ctx, - ) => T, + sandboxFn: (batchData: ReadonlyArray, ctx: Ctx) => T, ): R8RemoteNode { return new R8RemoteNode(deps, sandboxFn); } diff --git a/packages/ts/package.json b/packages/ts/package.json index 7c16d281..4fe12977 100644 --- a/packages/ts/package.json +++ b/packages/ts/package.json @@ -19,7 +19,10 @@ } } }, - "files": ["dist", "LICENSE"], + "files": [ + "dist", + "LICENSE" + ], "publishConfig": { "access": "public" }, diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index 09222f42..2a2bb26e 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -10,7 +10,7 @@ import { describe, expect, it } from "vitest"; import type { Ctx, Message } from "../index.js"; -import { node } from "../index.js"; +import { graph, node } from "../index.js"; const types = (msgs: Message[]) => msgs.map((m) => m[0]); function collect(n: { subscribe(s: (m: Message) => void): () => void }) { @@ -154,3 +154,61 @@ describe("C-5 PAUSE lockset multi-source (R-pause-lockset, R-pause-modes)", () = expect(n.cache).toBe(1); }); }); + +describe("C-6 synchronous feedback cycle → ERROR (R-reentrancy / D37)", () => { + it("rejects a sync feedback cycle as ERROR (no hang, no _pending desync)", () => { + // state S → derived D (=n+1) → effect E; E writes back to S, closing S→D→E→S. + const g = graph(); + const s = g.state(0); + const d = g.derived([s], (n) => (n as number) + 1); + const e = g.effect([d], (n) => { + s.set(n as number); // feedback → re-enters the wave + }); + let escaped = false; + try { + e.subscribe(() => {}); + } catch { + escaped = true; // the substrate throw must NOT escape — the graph layer catches it (D30) + } + expect(escaped).toBe(false); + // R-reentrancy: ERROR lands on a node ON the cycle — the value-level catch nearest the + // throw on the unwind (impl-determined, d or e), not necessarily the re-entered node. + expect([d.status, e.status]).toContain("errored"); + }); +}); + +describe("C-7 upstream control at a depless source (R-up-at-source / D38)", () => { + const make = () => { + const s = node([], null, { initial: 5 }); + const d = node([s], (ctx: Ctx) => + ctx.down([["DATA", ctx.depRecords[0].latest as number]]), + ); + const { msgs } = collect(d); + msgs.length = 0; + return { s, d, msgs }; + }; + + it("INVALIDATE-up is HONORED: source self-invalidates + cascades down", () => { + const { s, d, msgs } = make(); + d.up([["INVALIDATE"]]); // forwards to the depless terminus S → self _invalidate + expect(s.cache).toBeUndefined(); + expect(s.status).toBe("sentinel"); + expect(types(msgs)).toContain("INVALIDATE"); + expect(d.cache).toBeUndefined(); + }); + + it("DIRTY-up is DROPPED at the source (untouched, no down-cascade)", () => { + const { s, d, msgs } = make(); + d.up([["DIRTY"]]); + expect(s.cache).toBe(5); + expect(s.status).toBe("settled"); + expect(msgs).toEqual([]); + }); + + it("TEARDOWN-up is DROPPED at the source (not terminated, no down-cascade)", () => { + const { s, d, msgs } = make(); + d.up([["TEARDOWN"]]); + expect(s.status).toBe("settled"); + expect(msgs).toEqual([]); + }); +}); diff --git a/packages/ts/src/__tests__/graph.test.ts b/packages/ts/src/__tests__/graph.test.ts new file mode 100644 index 00000000..763b8214 --- /dev/null +++ b/packages/ts/src/__tests__/graph.test.ts @@ -0,0 +1,136 @@ +import { describe, expect, it } from "vitest"; +import type { Message } from "../index.js"; +import { graph } from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + n.subscribe((m) => msgs.push(m)); + return msgs; +} +const types = (msgs: Message[]) => msgs.map((m) => m[0]); + +describe("Graph — 8-verb sugar (CSP-2)", () => { + it("state + derived: value-level fn computes from dep values (D27)", () => { + const g = graph(); + const count = g.state(0); + const doubled = g.derived([count], (n) => n * 2); + collect(doubled); + expect(doubled.cache).toBe(0); + count.set(5); + expect(doubled.cache).toBe(10); + }); + + it("derived over multiple deps receives typed values in order", () => { + const g = graph(); + const a = g.state(2); + const b = g.state(3); + const sum = g.derived([a, b], (x, y) => x + y); + collect(sum); + expect(sum.cache).toBe(5); + b.set(10); + expect(sum.cache).toBe(12); + }); + + it("effect runs on dep settle and registers a deactivation cleanup", () => { + const g = graph(); + const s = g.state(1); + const seen: number[] = []; + let cleaned = 0; + const e = g.effect([s], (n) => { + seen.push(n); + return () => { + cleaned++; + }; + }); + const unsub = e.subscribe(() => {}); + expect(seen).toEqual([1]); + s.set(2); + expect(seen).toEqual([1, 2]); + unsub(); // last subscriber → deactivate → cleanup + expect(cleaned).toBe(1); + }); + + it("D30: a value-level fn that throws emits ERROR downstream (not a crash)", () => { + const g = graph(); + const s = g.state(1); + const bad = g.derived([s], (n) => { + if (n > 0) throw new Error("boom"); + return n; + }); + const msgs = collect(bad); + expect(types(msgs)).toContain("ERROR"); + expect(bad.status).toBe("errored"); + }); + + it("R-reentrancy via graph (D37): a synchronous feedback cycle yields ERROR, not a hang", () => { + const g = graph(); + const s = g.state(0); + const d = g.derived([s], (n) => n + 1); + const seen: Message[] = []; + // effect feeds back into s → S→D→E→S cycle. The substrate rejects the re-entry + // (throw); the graph layer's value-level boundary catches it → ERROR. No hang. + const e = g.effect([d], (n) => { + s.set(n as number); + }); + expect(() => e.subscribe((m) => seen.push(m))).not.toThrow(); // caught, not escaped + // R-reentrancy: the ERROR lands on a node ON the cycle — the value-level catch nearest + // the throw on the synchronous unwind (impl-determined, d or e), NOT necessarily the + // re-entered node. Assert SOME cycle node errored, not which (per the amended spec). + expect([d.status, e.status]).toContain("errored"); + }); +}); + +describe("Graph.describe — snapshot shape (R-describe / D39)", () => { + it("emits a flat snapshot with ids, factory names, status, value, deps, edges", () => { + const g = graph({ name: "demo" }); + const count = g.state(0, { name: "count" }); + const doubled = g.derived([count], (n) => n * 2, { name: "doubled" }); + collect(doubled); + count.set(5); + + const snap = g.describe(); + expect(snap.name).toBe("demo"); + const byId = Object.fromEntries(snap.nodes.map((n) => [n.id, n])); + expect(byId.count.factory).toBe("state"); + expect(byId.count.value).toBe(5); + expect(byId.count.status).toBe("settled"); + expect(byId.doubled.factory).toBe("derived"); + expect(byId.doubled.value).toBe(10); + expect(byId.doubled.deps).toEqual(["count"]); + expect(snap.edges).toContainEqual({ from: "count", to: "doubled" }); + }); + + it("absent value field = SENTINEL (never-emitted)", () => { + const g = graph(); + const s = g.state(0, { name: "s" }); + g.derived([s], (n) => n, { name: "d" }); // not subscribed → never runs + const snap = g.describe(); + const dNode = snap.nodes.find((n) => n.id === "d"); + expect(dNode && "value" in dNode).toBe(false); // SENTINEL → field absent + }); + + it("explain mode filters to the causal chain from→to", () => { + const g = graph(); + const a = g.state(1, { name: "a" }); + const b = g.derived([a], (x) => x + 1, { name: "b" }); + const c = g.derived([b], (x) => x + 1, { name: "c" }); + const side = g.state(9, { name: "side" }); // off the a→c chain + collect(c); + collect(g.derived([side], (x) => x, { name: "sideD" })); + + const snap = g.describe({ explain: { from: "a", to: "c" } }); + const ids = snap.nodes.map((n) => n.id).sort(); + expect(ids).toEqual(["a", "b", "c"]); + }); + + it("mount nests a subgraph under a :: prefixed path", () => { + const parent = graph({ name: "p" }); + parent.state(0, { name: "root" }); + const child = graph({ name: "c" }); + child.state(1, { name: "leaf" }); + parent.mount(child, { at: "sub" }); + + const snap = parent.describe(); + expect(snap.subgraphs?.[0].nodes.some((n) => n.id === "sub::leaf")).toBe(true); + }); +}); diff --git a/packages/ts/src/__tests__/inspect.test.ts b/packages/ts/src/__tests__/inspect.test.ts new file mode 100644 index 00000000..d2f8aa21 --- /dev/null +++ b/packages/ts/src/__tests__/inspect.test.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from "vitest"; +import type { ObserveEvent } from "../index.js"; +import { Dispatcher, graph } from "../index.js"; + +describe("observe — read-only enveloped egress (R-observe / D39)", () => { + it("streams ObserveEvents with path/msg/tier/seq; cached node pushes on subscribe", () => { + const g = graph(); + const count = g.state(0, { name: "count" }); + const doubled = g.derived([count], (n) => n * 2, { name: "doubled" }); + doubled.subscribe(() => {}); // activate so it has a cache + + const events: ObserveEvent[] = []; + const stop = g.observe("doubled").subscribe((e) => events.push(e)); + // push-on-subscribe: the cached DATA arrives immediately + expect(events.some((e) => e.path === "doubled" && e.msg[0] === "DATA")).toBe(true); + + count.set(5); + const data = events.filter((e) => e.msg[0] === "DATA").map((e) => e.msg[1]); + expect(data).toContain(10); + // envelope carries tier + a monotonic seq + const last = events[events.length - 1]; + expect(typeof last.seq).toBe("number"); + expect(last.tier).toBe(3); // DATA tier + stop(); + }); + + it("whole-graph observe (no path) taps every node; observe is NOT itself a node", () => { + const g = graph(); + const a = g.state(1, { name: "a" }); + g.derived([a], (n) => n + 1, { name: "b" }); + const events: ObserveEvent[] = []; + g.observe().subscribe((e) => events.push(e)); + a.set(2); + const paths = new Set(events.map((e) => e.path)); + expect(paths.has("a")).toBe(true); + expect(paths.has("b")).toBe(true); + // observing did not add a node to the graph (egress, not a node) + expect( + g + .describe() + .nodes.map((n) => n.id) + .sort(), + ).toEqual(["a", "b"]); + }); +}); + +describe("profile — dispatcher-backed counters, never on the thin node (R-profile / D39)", () => { + it("counts invokes per node when profiling is enabled (opt-in)", () => { + const g = graph({ dispatcher: new Dispatcher(), profile: true }); + const s = g.state(0, { name: "s" }); + const d = g.derived([s], (n) => n + 1, { name: "d" }); + d.subscribe(() => {}); + s.set(1); + s.set(2); + const p = g.profile(); + // d's fn ran: once on activation + once per set = 3 + expect(p.nodes.d.invokes).toBe(3); + expect(p.nodes.d.status).toBe("settled"); + // state node has no fn → no invokes + expect(p.nodes.s.invokes).toBe(0); + expect(p.totalInvokes).toBeGreaterThanOrEqual(3); + }); + + it("default graph does not record (zero overhead, F-PERF)", () => { + const g = graph({ dispatcher: new Dispatcher() }); // profile not enabled + const s = g.state(0, { name: "s" }); + const d = g.derived([s], (n) => n + 1, { name: "d" }); + d.subscribe(() => {}); + s.set(1); + expect(g.profile().nodes.d.invokes).toBe(0); // not recorded + expect(g.profile().totalInvokes).toBe(0); + }); +}); diff --git a/packages/ts/src/__tests__/operators.test.ts b/packages/ts/src/__tests__/operators.test.ts new file mode 100644 index 00000000..67429963 --- /dev/null +++ b/packages/ts/src/__tests__/operators.test.ts @@ -0,0 +1,97 @@ +import { describe, expect, it } from "vitest"; +import type { Message } from "../index.js"; +import { graph } from "../index.js"; + +const data = (msgs: Message[]) => + msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); + +describe("core operators (node sugar, D6/L1.5)", () => { + it("map emits fn(value)", () => { + const g = graph(); + const s = g.state(1); + const m = g.map(s, (n) => n * 2); + const msgs: Message[] = []; + m.subscribe((x) => msgs.push(x)); + expect(m.cache).toBe(2); + s.set(3); + expect(m.cache).toBe(6); + expect(data(msgs)).toEqual([2, 6]); + }); + + it("filter emits only when pred holds", () => { + const g = graph(); + const s = g.state(2); + const evens = g.filter(s, (n) => n % 2 === 0); + const msgs: Message[] = []; + evens.subscribe((x) => msgs.push(x)); + s.set(3); // filtered out + s.set(4); + expect(data(msgs)).toEqual([2, 4]); + expect(evens.cache).toBe(4); + }); + + it("scan accumulates with a seed", () => { + const g = graph(); + const s = g.state(1); + const sum = g.scan(s, (acc: number, v: number) => acc + v, 0); + const msgs: Message[] = []; + sum.subscribe((x) => msgs.push(x)); + s.set(2); + s.set(3); + expect(data(msgs)).toEqual([1, 3, 6]); + }); + + it("take emits the first n then COMPLETE (terminal-is-forever)", () => { + const g = graph(); + const s = g.state(1); + const first2 = g.take(s, 2); + const msgs: Message[] = []; + first2.subscribe((x) => msgs.push(x)); + s.set(2); // 2nd value → DATA + COMPLETE (a DIRTY precedes the DATA, two-phase) + s.set(3); // ignored (terminated) + expect(data(msgs)).toEqual([1, 2]); + expect(msgs[msgs.length - 1][0]).toBe("COMPLETE"); + expect(first2.status).toBe("completed"); + }); + + it("distinctUntilChanged suppresses repeats", () => { + const g = graph(); + const s = g.state(1); + const d = g.distinctUntilChanged(s); + const msgs: Message[] = []; + d.subscribe((x) => msgs.push(x)); + s.set(1); // same → suppressed + s.set(2); + s.set(2); // same → suppressed + s.set(3); + expect(data(msgs)).toEqual([1, 2, 3]); + }); + + it("merge interleaves several sources (partial — fires on any)", () => { + const g = graph(); + const a = g.state(1); + const b = g.state(2); + const m = g.merge([a, b]); + const vals: unknown[] = []; + m.subscribe((msg) => { + if (msg[0] === "DATA") vals.push(msg[1]); + }); + expect(vals).toContain(1); + expect(vals).toContain(2); + a.set(10); + expect(vals).toContain(10); + b.set(20); + expect(vals).toContain(20); + }); + + it("describe shows the real operator factory name (D6), not 'derived'", () => { + const g = graph(); + const s = g.state(0, { name: "s" }); + g.map(s, (n) => n + 1, { name: "inc" }); + g.filter(s, (n) => n > 0, { name: "pos" }); + const snap = g.describe(); + const byId = Object.fromEntries(snap.nodes.map((n) => [n.id, n])); + expect(byId.inc.factory).toBe("map"); + expect(byId.pos.factory).toBe("filter"); + }); +}); diff --git a/packages/ts/src/__tests__/reentrancy.test.ts b/packages/ts/src/__tests__/reentrancy.test.ts new file mode 100644 index 00000000..8395bec0 --- /dev/null +++ b/packages/ts/src/__tests__/reentrancy.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx } from "../index.js"; +import { node } from "../index.js"; + +const plus1 = (ctx: Ctx) => ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 1]]); + +describe("R-reentrancy (D37) — synchronous feedback cycle is rejected", () => { + it("a fn that re-drives its own dep mid-wave throws (cycle detected node-locally)", () => { + // state S → derived D (=n+1) → effect E; E feeds back into S, closing S→D→E→S. + // Activating the chain drives the cycle synchronously and must reject. + const s = node([], null, { initial: 0 }); + const d = node([s], plus1); + const e = node([d], (ctx: Ctx) => { + // feedback write back to S (an indirect dep of E) — re-enters the wave. + s.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + expect(() => e.subscribe(() => {})).toThrow(/feedback cycle|R-reentrancy/i); + }); + + it("an acyclic chain does NOT throw (the guard does not false-positive)", () => { + const s = node([], null, { initial: 1 }); + const d = node([s], plus1); + const e = node([d], (ctx: Ctx) => + ctx.down([["DATA", ctx.depRecords[0].latest as number]]), + ); + expect(() => e.subscribe(() => {})).not.toThrow(); + expect(e.cache).toBe(2); + }); + + it("a rejected cycle leaves other graphs usable (try/finally resets the flag on unwind)", () => { + const s = node([], null, { initial: 0 }); + const d = node([s], plus1); + const e = node([d], (ctx: Ctx) => { + s.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + expect(() => e.subscribe(() => {})).toThrow(); + // A fresh, independent chain still computes — no node left with a stuck in-wave flag. + const a = node([], null, { initial: 7 }); + const b = node([a], plus1); + b.subscribe(() => {}); + expect(b.cache).toBe(8); + }); +}); diff --git a/packages/ts/src/__tests__/up-source.test.ts b/packages/ts/src/__tests__/up-source.test.ts new file mode 100644 index 00000000..719e7b6c --- /dev/null +++ b/packages/ts/src/__tests__/up-source.test.ts @@ -0,0 +1,78 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { batch, node } from "../index.js"; + +// depless source S (cached) → derived D (passthrough) → sink. D originates ctx.up([msg]), +// which forwards upstream to the terminus S (R-up-at-source / D38). +function setup() { + const s = node([], null, { initial: 5 }); + const d = node([s], (ctx: Ctx) => + ctx.down([["DATA", ctx.depRecords[0].latest as number]]), + ); + const seen: Message[] = []; + d.subscribe((m) => seen.push(m)); + return { s, d, seen }; +} + +describe("R-up-at-source (D38) — upstream control at a depless source", () => { + it("INVALIDATE-up is HONORED: source self-invalidates + cascades down", () => { + const { s, d, seen } = setup(); + expect(s.cache).toBe(5); + seen.length = 0; + d.up([["INVALIDATE"]]); // → forwards to S (terminus) → self _invalidate → cascade down + expect(s.cache).toBeUndefined(); // SENTINEL + expect(s.status).toBe("sentinel"); + expect(seen.map((m) => m[0])).toContain("INVALIDATE"); // D observed the down-cascade + expect(d.cache).toBeUndefined(); + }); + + it("INVALIDATE-up fires the source's onInvalidate hook once", () => { + let flushed = 0; + // a source with a fn so it can register onInvalidate (producer-style depless node) + const s = node([], (ctx: Ctx) => { + ctx.onInvalidate(() => { + flushed++; + }); + ctx.down([["DATA", 9]]); + }); + const d = node([s], (ctx: Ctx) => + ctx.down([["DATA", ctx.depRecords[0].latest as number]]), + ); + d.subscribe(() => {}); + expect(s.cache).toBe(9); + d.up([["INVALIDATE"]]); + expect(flushed).toBe(1); + expect(s.cache).toBeUndefined(); + }); + + it("DIRTY-up is DROPPED at the source: untouched, no down-cascade", () => { + const { s, d, seen } = setup(); + seen.length = 0; + d.up([["DIRTY"]]); // D forwards DIRTY to S; S is the depless terminus → drop + expect(s.cache).toBe(5); + expect(s.status).toBe("settled"); + expect(seen.length).toBe(0); // nothing propagated downstream + }); + + it("TEARDOWN-up is DROPPED at the source: not terminated, no down-cascade", () => { + const { s, d, seen } = setup(); + seen.length = 0; + d.up([["TEARDOWN"]]); // forwarded to S → terminus → drop + expect(s.status).toBe("settled"); // not terminated + expect(s.cache).toBe(5); + expect(seen.length).toBe(0); + }); + + it("INVALIDATE-up defers inside a batch, cascades on commit (A-2: routed via _down)", () => { + const s = node([], null, { initial: 5 }); + const d = node([s], (ctx: Ctx) => + ctx.down([["DATA", ctx.depRecords[0].latest as number]]), + ); + d.subscribe(() => {}); + batch(() => { + d.up([["INVALIDATE"]]); // forwarded to depless S → S._down → tier-4 deferred + expect(s.cache).toBe(5); // deferred — NOT yet applied mid-batch + }); + expect(s.cache).toBeUndefined(); // committed → source self-invalidated + }); +}); diff --git a/packages/ts/src/dispatcher/index.ts b/packages/ts/src/dispatcher/index.ts index 21c6ef5c..0862545b 100644 --- a/packages/ts/src/dispatcher/index.ts +++ b/packages/ts/src/dispatcher/index.ts @@ -66,16 +66,49 @@ class LocalAsyncPool implements Pool { * First-class dispatcher (D21). Owns pools; graph binds to one (default = process-global, * D26 — the only global singleton). Pool trait is pluggable for WorkerPool/RemotePool (D20). */ +/** Per-handle profiling counters (D39 / R-profile). Lives on the dispatcher — the invoke + * funnel (F-DISPATCH-ALL) — NEVER on the thin node (R-node-thin). */ +export interface HandleStat { + invokes: number; + totalDurationNs: number; + lastDurationNs: number; +} + +const statKey = (h: Handle): string => `${h.poolId}:${h.handleId}`; + export class Dispatcher { private pools: Pool[] = []; readonly syncPoolId: number; readonly asyncPoolId: number; + // opt-in profile recorder (default OFF → zero overhead, F-PERF). + private _recording = false; + private _stats = new Map(); + private _totalInvokes = 0; + constructor() { this.syncPoolId = this.addPool(new LocalSyncPool()); this.asyncPoolId = this.addPool(new LocalAsyncPool()); } + /** Turn the profile recorder on/off (D39). Off = zero overhead on invoke. */ + setRecording(on: boolean): void { + this._recording = on; + } + /** Reset accumulated profiling counters. */ + clearStats(): void { + this._stats.clear(); + this._totalInvokes = 0; + } + /** Read a handle's accumulated counters (undefined if it never ran while recording). */ + statFor(handle: Handle): HandleStat | undefined { + return this._stats.get(statKey(handle)); + } + /** Total fn invocations recorded across the dispatcher. */ + get totalInvokes(): number { + return this._totalInvokes; + } + addPool(pool: Pool): number { const id = this.pools.length; this.pools.push(pool); @@ -91,7 +124,27 @@ export class Dispatcher { /** Uniform sync-void invoke (R-sync-core / R-dispatch-all). */ invoke(handle: Handle, ctx: Ctx): void { - this.pools[handle.poolId].invoke(handle.handleId, ctx); + if (!this._recording) { + this.pools[handle.poolId].invoke(handle.handleId, ctx); + return; + } + this._totalInvokes++; + const t0 = performance.now(); + try { + this.pools[handle.poolId].invoke(handle.handleId, ctx); + } finally { + const dur = (performance.now() - t0) * 1e6; // ms → ns + const key = statKey(handle); + const s = this._stats.get(key) ?? { + invokes: 0, + totalDurationNs: 0, + lastDurationNs: 0, + }; + s.invokes++; + s.lastDurationNs = dur; + s.totalDurationNs += dur; + this._stats.set(key, s); + } } poolKind(poolId: number): PoolKind { diff --git a/packages/ts/src/graph/describe.ts b/packages/ts/src/graph/describe.ts new file mode 100644 index 00000000..8ed36054 --- /dev/null +++ b/packages/ts/src/graph/describe.ts @@ -0,0 +1,43 @@ +/** + * describe() snapshot shape (R-describe / D39). + * + * Static, JSON-serializable structure snapshot. Renderers (pretty/mermaid/d2) are + * pure functions over this shape — NOT methods. Per-language (D24, never in parity); + * this documents the cross-lang contract. + */ + +import type { Status } from "../node/node.js"; + +export interface DescribeNode { + /** Stable mount-aware `::` path (auto-numbered when unnamed). Edge key. */ + id: string; + /** Optional debug name. */ + name?: string; + /** Operator/verb real name (D6/L1.5 — "map"/"state", not "derived"). */ + factory: string; + /** R-status-enum (7). */ + status: Status; + /** Cache snapshot at call time; field ABSENT = SENTINEL / never-emitted. */ + value?: unknown; + /** Dep ids (R-edges-derived: edges are a pure fn of deps). */ + deps: string[]; + /** Static annotations attached via g.* opts (R-meta-presentation). */ + meta?: Record; +} + +export interface DescribeEdge { + from: string; + to: string; +} + +export interface DescribeSnapshot { + name?: string; + nodes: DescribeNode[]; + edges: DescribeEdge[]; + subgraphs?: DescribeSnapshot[]; +} + +export interface DescribeOpts { + /** Causal-chain mode: filter to nodes on a path from→to (R-describe). */ + explain?: { from: string; to: string }; +} diff --git a/packages/ts/src/graph/graph.ts b/packages/ts/src/graph/graph.ts new file mode 100644 index 00000000..41d02eb0 --- /dev/null +++ b/packages/ts/src/graph/graph.ts @@ -0,0 +1,469 @@ +/** + * The Graph layer (CSP-2): convenience + inspection entry (D5 / R-graph-role). + * + * g.* sugar desugars to dispatcher.register + Node and auto-registers into the graph + * inspection index (D27: sugar→ctx-fn conversion lives strictly here; the substrate node + * only ever sees `(ctx)=>void`). The Graph owns naming, describe, mount, and the D30 + * value-level throw→ERROR boundary. The bare node()/dispatcher stay inspection-free + * (R-node-thin). + * + * Canonical authority: ~/src/graphrefly — D3/D5/D27/D30/D36/D37/D39, R-graph-role, + * R-describe, R-fn-contract, R-edges-derived, R-meta-presentation. + */ + +import { type BatchCtx, batch as batchRun } from "../batch/batch.js"; +import type { Ctx, NodeFn } from "../ctx/types.js"; +import { type Dispatcher, defaultDispatcher } from "../dispatcher/index.js"; +import { Node, type NodeOptions } from "../node/node.js"; +import { messageTier } from "../protocol/messages.js"; +import type { DescribeEdge, DescribeNode, DescribeOpts, DescribeSnapshot } from "./describe.js"; +import type { NodeProfile, ObserveStream, Profile } from "./inspect.js"; + +/** Map a tuple of Nodes to the tuple of their value types (typed value-level fn args). */ +type DepValues[]> = { + [K in keyof D]: D[K] extends Node ? V : never; +}; + +/** Value-level derived fn: receives dep values, returns the next value (undefined = no emit). */ +export type DerivedFn[], T> = ( + ...values: DepValues +) => T | undefined; + +/** Value-level effect fn: receives dep values, optionally returns a deactivation cleanup. */ +export type EffectFn[]> = ( + ...values: DepValues + // biome-ignore lint/suspicious/noConfusingVoidType: effect returns void OR a cleanup fn — the void arm keeps `(v) => { sideEffect(v) }` ergonomic (React EffectCallback idiom); dropping it would force an explicit `return undefined`. +) => void | (() => void); + +/** Sugar options — node options minus dispatcher (graph owns it) plus naming/meta. */ +export interface SugarOpts extends Omit, "dispatcher"> { + name?: string; + meta?: Record; +} + +export interface GraphOptions { + name?: string; + /** Bind to a dispatcher (default = process-global, D26). */ + dispatcher?: Dispatcher; + /** + * Turn on the dispatcher profile recorder (D39 / F-PERF default off). NOTE: this + * switches recording on for the WHOLE bound dispatcher (the default is process-global, + * D26) — every graph sharing it then pays the recorder cost. For isolated profiling + * use a dedicated dispatcher: `graph({ dispatcher: new Dispatcher(), profile: true })`. + */ + profile?: boolean; +} + +interface Entry { + node: Node; + id: string; + name?: string; + factory: string; + deps: readonly Node[]; + meta?: Record; +} + +/** + * A state node (L4-Q1): a manual source whose `.set(v)` is `node.down([[DATA, v]])` sugar. + * Extends the substrate Node so it is usable as a dep AND carries the graph-layer `.set`. + */ +export class StateNode extends Node { + set(v: T): void { + this.down([["DATA", v]]); + } +} + +export class Graph { + readonly name?: string; + private readonly _dispatcher: Dispatcher; + private readonly _entries = new Map, Entry>(); + private readonly _byId = new Map>(); + private readonly _mounts: Array<{ at: string; graph: Graph }> = []; + private _seq = 0; + private _clock = 0; // graph-local monotonic clock for observe seq (D26) + + constructor(opts: GraphOptions = {}) { + this.name = opts.name; + this._dispatcher = opts.dispatcher ?? defaultDispatcher; + if (opts.profile) this._dispatcher.setRecording(true); + } + + // ── registration / inspection index ── + + private _add( + n: Node, + factory: string, + deps: readonly Node[], + opts: SugarOpts, + ): Node { + const id = opts.name ?? `${factory}#${this._seq++}`; + this._entries.set(n as Node, { + node: n as Node, + id, + name: opts.name, + factory, + deps, + meta: opts.meta, + }); + this._byId.set(id, n as Node); + return n; + } + + private _nodeOpts(opts: SugarOpts): NodeOptions { + // strip graph-only fields; inject the bound dispatcher + const { name: _n, meta: _m, ...rest } = opts; + return { ...rest, dispatcher: this._dispatcher }; + } + + /** Look up a registered node by its id. */ + find(id: string): Node | undefined { + return this._byId.get(id); + } + + // ── 8 verbs (core: node/state/batch + sugar: producer/derived/effect/mount) ── + + /** ctx-level power surface: a raw `(ctx)=>void` fn (or a passthrough/state when null). */ + node( + deps: readonly Node[] = [], + fn: NodeFn | null = null, + opts: SugarOpts = {}, + ): Node { + const n = new Node(deps as Node[], fn, this._nodeOpts(opts)); + return this._add(n, "node", deps, opts); + } + + /** A manual source with `.set(v)` (L4-Q1). */ + state(initial: T, opts: SugarOpts = {}): StateNode { + const n = new StateNode([], null, { ...this._nodeOpts(opts), initial }); + this._add(n, "state", [], opts); + return n; + } + + /** ctx-level depless source; its fn runs on activation (R-rom-ram). */ + producer(fn: NodeFn, opts: SugarOpts = {}): Node { + const n = new Node([], fn, this._nodeOpts(opts)); + return this._add(n, "producer", [], opts); + } + + /** value-level pure transform: deps → value (D27 wrapped; D30 throw→ERROR). */ + derived[], T>( + deps: D, + fn: DerivedFn, + opts: SugarOpts = {}, + ): Node { + const ctxFn: NodeFn = (ctx: Ctx) => { + try { + const args = ctx.depRecords.map((r) => r.latest) as DepValues; + const result = fn(...args); + if (result !== undefined) ctx.down([["DATA", result]]); + } catch (e) { + ctx.down([["ERROR", e]]); // D30: value-level throw → graph-layer ERROR + } + }; + const n = new Node([...deps], ctxFn, this._nodeOpts(opts)); + return this._add(n, "derived", deps, opts); + } + + /** value-level sink: deps → effect; return value (a fn) becomes onDeactivation (D28). */ + effect[]>( + deps: D, + fn: EffectFn, + opts: SugarOpts = {}, + ): Node { + // effect cleanup = deactivation-only (D28 / Flag 3): the LATEST returned cleanup + // fires ONCE when the effect deactivates (not between re-runs — onRerun was cut). + // State lives in ctx.state (per-node, wiped on fresh-lifecycle) so the deactivation + // hook is registered exactly once per activation and re-registers after a re-activation. + interface EffectState { + registered: boolean; + cleanup?: () => void; + } + const ctxFn: NodeFn = (ctx: Ctx) => { + try { + const args = ctx.depRecords.map((r) => r.latest) as DepValues; + const cleanup = fn(...args); + const st: EffectState = ctx.state.get() ?? { registered: false }; + st.cleanup = typeof cleanup === "function" ? cleanup : undefined; + if (!st.registered) { + st.registered = true; + ctx.onDeactivation(() => st.cleanup?.()); + } + ctx.state.set(st); + } catch (e) { + ctx.down([["ERROR", e]]); + } + }; + const n = new Node([...deps], ctxFn, this._nodeOpts(opts)); + return this._add(n, "effect", deps, opts); + } + + /** Declarative batch (D12): one wave, success→commit / throw→rollback. */ + batch(fn: (bctx: BatchCtx) => T): T { + return batchRun(fn); + } + + /** Embed a child graph addressable under `at` (R-mount; mount has no deps). */ + mount(child: Graph, opts: { at: string }): void { + this._mounts.push({ at: opts.at, graph: child }); + } + + // ── operators (node sugar, D6/L1.5; describe shows the real factory name) ── + // Core set per L4-Q7 (main package); time-based + higher-order operators are a + // separate subpackage. Each is value-level over the node primitive; the shared _op + // wrapper carries the D30 throw→ERROR boundary. + + private _op( + deps: readonly Node[], + factory: string, + body: (ctx: Ctx) => void, + opts: SugarOpts, + ): Node { + const ctxFn: NodeFn = (ctx: Ctx) => { + try { + body(ctx); + } catch (e) { + ctx.down([["ERROR", e]]); + } + }; + const n = new Node([...deps], ctxFn, this._nodeOpts(opts)); + return this._add(n, factory, deps, opts); + } + + /** map: emit fn(value). */ + map(src: Node, fn: (v: S) => T, opts: SugarOpts = {}): Node { + return this._op( + [src as Node], + "map", + (ctx) => { + ctx.down([["DATA", fn(ctx.depRecords[0].latest as S)]]); + }, + opts, + ); + } + + /** filter: emit value only when pred(value) (else skip the wave). */ + filter(src: Node, pred: (v: S) => boolean, opts: SugarOpts = {}): Node { + return this._op( + [src as Node], + "filter", + (ctx) => { + const v = ctx.depRecords[0].latest as S; + if (pred(v)) ctx.down([["DATA", v]]); + }, + opts, + ); + } + + /** scan: stateful accumulator over the upstream (acc seeded once, kept in ctx.state). */ + scan( + src: Node, + reducer: (acc: T, v: S) => T, + seed: T, + opts: SugarOpts = {}, + ): Node { + return this._op( + [src as Node], + "scan", + (ctx) => { + const acc = ctx.state.get() ?? seed; + const next = reducer(acc, ctx.depRecords[0].latest as S); + ctx.state.set(next); + ctx.down([["DATA", next]]); + }, + opts, + ); + } + + /** take: emit the first n values, then COMPLETE (terminal-is-forever). */ + take(src: Node, n: number, opts: SugarOpts = {}): Node { + return this._op( + [src as Node], + "take", + (ctx) => { + if (n <= 0) { + ctx.down([["COMPLETE"]]); + return; + } + const count = ctx.state.get() ?? 0; + if (count >= n) return; // already satisfied + const v = ctx.depRecords[0].latest as S; + const next = count + 1; + ctx.state.set(next); + ctx.down(next >= n ? [["DATA", v], ["COMPLETE"]] : [["DATA", v]]); + }, + opts, + ); + } + + /** distinctUntilChanged: emit only when the value differs from the previous emit. */ + distinctUntilChanged( + src: Node, + eq: (a: S, b: S) => boolean = Object.is, + opts: SugarOpts = {}, + ): Node { + return this._op( + [src as Node], + "distinctUntilChanged", + (ctx) => { + const v = ctx.depRecords[0].latest as S; + const prev = ctx.state.get<{ v: S }>(); + if (prev && eq(prev.v, v)) return; + ctx.state.set({ v }); + ctx.down([["DATA", v]]); + }, + opts, + ); + } + + /** merge: interleave several sources — emit each DATA from whichever dep fired (partial). */ + merge(srcs: readonly Node[], opts: SugarOpts = {}): Node { + return this._op( + srcs as readonly Node[], + "merge", + (ctx) => { + for (const r of ctx.depRecords) { + if (r.batch && r.batch.length > 0) { + for (const v of r.batch) ctx.down([["DATA", v]]); + } + } + }, + { ...opts, partial: true }, + ); + } + + // ── inspection: describe (Slice B); observe/profile land in Slice D ── + + /** Static structure snapshot (R-describe / D39). `_prefix` carries the mount path. */ + describe(opts: DescribeOpts = {}, _prefix = ""): DescribeSnapshot { + const localId = (n: Node): string => { + const e = this._entries.get(n); + return e ? `${_prefix}${e.id}` : `${_prefix}?`; + }; + const nodes: DescribeNode[] = []; + const edges: DescribeEdge[] = []; + for (const entry of this._entries.values()) { + const id = `${_prefix}${entry.id}`; + const dnode: DescribeNode = { + id, + factory: entry.factory, + status: entry.node.status, + deps: entry.deps.map(localId), + }; + if (entry.name !== undefined) dnode.name = entry.name; + if (entry.node.cache !== undefined) dnode.value = entry.node.cache; // absent = SENTINEL + if (entry.meta !== undefined) dnode.meta = entry.meta; + nodes.push(dnode); + for (const d of entry.deps) edges.push({ from: localId(d), to: id }); + } + const snap: DescribeSnapshot = { nodes, edges }; + if (this.name !== undefined) snap.name = this.name; + if (this._mounts.length > 0) { + snap.subgraphs = this._mounts.map((m) => m.graph.describe({}, `${_prefix}${m.at}::`)); + } + return opts.explain ? explainSubset(snap, opts.explain) : snap; + } + + private _observeTargets(path?: string): Array<[string, Node]> { + const all: Array<[string, Node]> = [...this._entries.values()].map((e) => [ + e.id, + e.node, + ]); + if (path === undefined) return all; // whole graph + const exact = this._byId.get(path); + if (exact) return [[path, exact]]; // single node + return all.filter(([id]) => id.startsWith(`${path}::`)); // subtree (mount-aware :: boundary, QA A-1) + } + + /** + * observe(path?) = read-only enveloped EGRESS (R-observe / D39). NOT a graph node — it + * taps the target node(s) via subscribe and forwards each Message as an ObserveEvent. + * No path = whole graph; an exact id = a single node; otherwise a `::`-prefix subtree. + */ + // NOTE (R-observe/D19): observe is a real, lazily-ACTIVATING subscriber — observing a + // cold node runs its fn (and activates upstream); whole-graph observe() activates the + // graph. "Read-only" means it never emits/mutates node state, NOT that it avoids + // activation. Use it knowing inspection of a cold graph wakes it. + observe(path?: string): ObserveStream { + const targets = this._observeTargets(path); + return { + subscribe: (sink) => { + const unsubs = targets.map(([id, n]) => + n.subscribe((msg) => { + sink({ path: id, msg, tier: messageTier(msg[0]), seq: this._clock++ }); + }), + ); + return () => { + for (const u of unsubs) u(); + }; + }, + }; + } + + /** + * profile() = accumulated-counter snapshot (R-profile / D39). invokes + duration are + * dispatcher-backed (the invoke funnel, F-DISPATCH-ALL) — counters never live on the + * thin node (R-node-thin). Requires `graph({ profile: true })` (opt-in, F-PERF). + */ + profile(): Profile { + const nodes: Record = {}; + let totalInvokes = 0; + for (const e of this._entries.values()) { + const h = e.node.handle; + const stat = h ? this._dispatcher.statFor(h) : undefined; + const invokes = stat?.invokes ?? 0; + nodes[e.id] = { + invokes, + totalDurationNs: stat?.totalDurationNs ?? 0, + lastDurationNs: stat?.lastDurationNs ?? 0, + status: e.node.status, + }; + // graph-scoped (QA D-2): sum THIS graph's nodes, NOT the dispatcher-global + // counter (which would leak invokes from other graphs sharing the dispatcher). + totalInvokes += invokes; + } + return { totalInvokes, nodes }; + } +} + +/** Filter a snapshot to the causal chain from→to (forward-reachable(from) ∩ back-reachable(to)). */ +function explainSubset( + snap: DescribeSnapshot, + chain: { from: string; to: string }, +): DescribeSnapshot { + const fwd = new Map(); + const rev = new Map(); + const push = (map: Map, k: string, v: string): void => { + const a = map.get(k); + if (a) a.push(v); + else map.set(k, [v]); + }; + for (const e of snap.edges) { + push(fwd, e.from, e.to); + push(rev, e.to, e.from); + } + const reach = (start: string, adj: Map): Set => { + const seen = new Set([start]); + const stack = [start]; + while (stack.length > 0) { + const cur = stack.pop() as string; + for (const nxt of adj.get(cur) ?? []) { + if (!seen.has(nxt)) { + seen.add(nxt); + stack.push(nxt); + } + } + } + return seen; + }; + const onPath = new Set([...reach(chain.from, fwd)].filter((id) => reach(chain.to, rev).has(id))); + return { + ...(snap.name !== undefined ? { name: snap.name } : {}), + nodes: snap.nodes.filter((n) => onPath.has(n.id)), + edges: snap.edges.filter((e) => onPath.has(e.from) && onPath.has(e.to)), + }; +} + +/** Construct a Graph (default global dispatcher; L4-Q4). */ +export function graph(opts: GraphOptions = {}): Graph { + return new Graph(opts); +} diff --git a/packages/ts/src/graph/inspect.ts b/packages/ts/src/graph/inspect.ts new file mode 100644 index 00000000..2afd4e2d --- /dev/null +++ b/packages/ts/src/graph/inspect.ts @@ -0,0 +1,41 @@ +/** + * observe / profile inspection shapes (R-observe, R-profile / D39). + * + * observe = a read-only enveloped EGRESS (not a graph node). profile = an opt-in + * accumulated-counter snapshot backed by the dispatcher recorder (counters never live on + * the thin node, R-node-thin). Per-language (D24, never in parity). + */ + +import type { Status } from "../node/node.js"; +import type { Message } from "../protocol/messages.js"; + +/** One item of an observe() stream. The envelope carries node identity + ordering. */ +export interface ObserveEvent { + /** Mount-aware `::` path of the emitting node (shared key with describe/profile). */ + path: string; + /** The protocol message observed at that node. */ + msg: Message; + /** messageTier(msg[0]). */ + tier: number; + /** Graph-local monotonic sequence (D26) — orders events across nodes. */ + seq: number; +} + +/** A read-only egress: subscribe to the live ObserveEvent stream; call the returned fn to stop. */ +export interface ObserveStream { + subscribe(sink: (e: ObserveEvent) => void): () => void; +} + +/** Per-node profile counters (D39). invokes/duration are dispatcher-backed (R-profile). */ +export interface NodeProfile { + invokes: number; + totalDurationNs: number; + lastDurationNs: number; + status: Status; +} + +/** profile() snapshot. Shares the mount-aware `::` path key with describe/observe. */ +export interface Profile { + totalInvokes: number; + nodes: Record; +} diff --git a/packages/ts/src/index.ts b/packages/ts/src/index.ts index 7f59bf53..363f5476 100644 --- a/packages/ts/src/index.ts +++ b/packages/ts/src/index.ts @@ -1,9 +1,9 @@ /** - * @graphrefly/ts — clean-slate TypeScript substrate (CSP-1 kernel). + * @graphrefly/ts — clean-slate TypeScript package. * - * Canonical authority: ~/src/graphrefly/spec/rules.jsonl + decisions.jsonl (D1-D36). - * Scope: node / dispatcher / pool / wave protocol. The graph layer, 8-verb sugar, - * operators, and inspection are CSP-2. + * Canonical authority: ~/src/graphrefly/spec/rules.jsonl + decisions.jsonl. + * CSP-1 substrate: node / dispatcher / pool / wave protocol. + * CSP-2 graph layer: Graph + 8-verb sugar + describe/observe/profile inspection. */ export { type BatchCtx, batch } from "./batch/batch.js"; @@ -12,8 +12,25 @@ export { Dispatcher, defaultDispatcher, type Handle, + type HandleStat, type Pool, type PoolKind, } from "./dispatcher/index.js"; +export type { + DescribeEdge, + DescribeNode, + DescribeOpts, + DescribeSnapshot, +} from "./graph/describe.js"; +export { + type DerivedFn, + type EffectFn, + Graph, + type GraphOptions, + graph, + StateNode, + type SugarOpts, +} from "./graph/graph.js"; +export type { NodeProfile, ObserveEvent, ObserveStream, Profile } from "./graph/inspect.js"; export { dynamicNode, Node, type NodeOptions, node, type Status } from "./node/node.js"; export * from "./protocol/messages.js"; diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index 43e62d3f..b1aee710 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -180,6 +180,16 @@ export class Node { return this._status; } + /** + * The fn handle (pure data `(poolId, handleId)`, D7) or null for state/passthrough + * nodes. Inspection-only (L1.6 handle is referenceable/inspectable) — lets the graph + * layer key a dispatcher-backed profile recorder WITHOUT putting counters on the node + * (R-node-thin / D39). + */ + get handle(): Handle | null { + return this._handle; + } + /** R-push-subscribe: a new sink receives START, then cached DATA (or DIRTY if dirty). */ subscribe(sink: Sink): () => void { // R-terminal: late subscribe to a terminal node either resets (resubscribable) @@ -427,11 +437,24 @@ export class Node { } private _runWave(): void { + // R-reentrancy (D37): a fn that re-drives its own dep mid-wave re-enters here while + // _insideRunWave is still set — a synchronous feedback cycle. Reject (throw); the graph + // layer catches it and converts to [[ERROR, e]] (D30). The try/finally resets the flag + // on every frame as the throw unwinds, leaving the graph clean for the catch. Detection + // is node-local and free — it reuses the existing _insideRunWave flag (no new structure, + // dispatcher stays a pure funnel). + if (this._insideRunWave) + throw new Error( + "synchronous feedback cycle: node fn re-entered its own wave (R-reentrancy / D37)", + ); this._hasCalledFnOnce = true; const ctx = this._buildCtx(); this._insideRunWave = true; - this._dispatcher.invoke(this._handle as Handle, ctx); - this._insideRunWave = false; + try { + this._dispatcher.invoke(this._handle as Handle, ctx); + } finally { + this._insideRunWave = false; + } // roll wave-local state forward for (let i = 0; i < this._depBatch.length; i++) this._depBatch[i] = null; @@ -651,8 +674,18 @@ export class Node { this._pauseAcquire(m[1]); } else if (m[0] === "RESUME") { this._pauseRelease(m[1]); + } else if (this._deps.length === 0) { + // R-up-at-source (D38): a depless source is the terminus of upstream control. + // INVALIDATE → HONOR the invalidate-request. Routed through _down (NOT a direct + // _invalidate call, QA A-2) so the invalidate-request respects batch-defer (D12) + // and pause-buffer exactly like a downstream-originated INVALIDATE; _down's + // INVALIDATE branch calls _invalidate() (clear cache → SENTINEL, fire onInvalidate, + // broadcast downstream). Outside batch/pause it is identical to a direct call. + // DIRTY / TEARDOWN → DROP (no coherent terminus action; self-dirty would wedge + // downstream awaiting a settle that never comes; source lifecycle is source-owned). + if (m[0] === "INVALIDATE") this._down([["INVALIDATE"]]); } else { - // DIRTY / INVALIDATE / TEARDOWN forward upstream toward deps (observers react). + // dep-bearing intermediate: forward upstream toward deps, no self-action. for (const dep of this._deps) dep.up([m]); } } From 05d15b4813864c6bf4913868728eb02ab890d3b0 Mon Sep 17 00:00:00 2001 From: David Chen Date: Thu, 28 May 2026 21:50:38 -0700 Subject: [PATCH 004/175] chore: update conformance and design-review SKILL documentation - Removed the `disable-model-invocation` field from both `conformance` and `design-review` SKILL documentation for clarity. - Updated descriptions to enhance understanding of the behavioral checks and validation processes in the respective skills. --- .claude/skills/conformance/SKILL.md | 1 - .claude/skills/design-review/SKILL.md | 1 - packages/ts/src/__tests__/conformance.test.ts | 20 +++++++++++++++---- 3 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.claude/skills/conformance/SKILL.md b/.claude/skills/conformance/SKILL.md index 133c7d1a..2acd3479 100644 --- a/.claude/skills/conformance/SKILL.md +++ b/.claude/skills/conformance/SKILL.md @@ -1,7 +1,6 @@ --- name: conformance description: "Behavioral conformance check across GraphReFly language runtimes (ts/rust/py) for the clean-slate redesign. Replaces the old structural 'parity' diff. Parity = does each runtime satisfy the wave-protocol behavior (conformance scenarios) + dispatcher contract — NOT 'do the symbol sets match'. Use after implementing/changing substrate behavior in any runtime, or when adding a new protocol rule. Authors/runs language-agnostic scenarios and updates conformance.jsonl runtime status. Triggers: 'conformance', 'cross-lang check', 'does rust match', 'parity', 'run the conformance suite', 'is the substrate behavior consistent'." -disable-model-invocation: true argument-hint: "[rule-id | scenario-id | 'full'] [optional: runtime ts|rust|py]" --- diff --git a/.claude/skills/design-review/SKILL.md b/.claude/skills/design-review/SKILL.md index e96e9a0c..4744f96e 100644 --- a/.claude/skills/design-review/SKILL.md +++ b/.claude/skills/design-review/SKILL.md @@ -1,7 +1,6 @@ --- name: design-review description: "Validate the design of a new primitive or API surface against the 5-question lens (Q5–Q9 from the per-unit review format). Use BEFORE coding (or right after a sketch lands) when adding a new public API / pattern factory / domain primitive. Triggers: 'design review', 'review the design', 'is this the right shape', 'before I implement'. Different from /qa — that finds bugs in landed code; this validates abstraction + long-term shape + reactive composability + alternatives." -disable-model-invocation: true argument-hint: "[ | | --diff] [optional context]" --- diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index 2a2bb26e..dcf143ab 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -87,8 +87,8 @@ describe("C-3 INVALIDATE × ctx.state × onInvalidate (R-invalidate-idempotent, }); }); -describe("C-4 mixed sync/async diamond (R-diamond, R-two-phase, R-first-run-gate)", () => { - it("joins exactly once after BOTH the sync and async legs settle", () => { +describe("C-4 mixed sync/async diamond (R-diamond, R-two-phase, R-first-run-gate, R-dirty-before-data)", () => { + it("joins exactly once after BOTH legs settle, re-emitting DIRTY before DATA on the next wave", () => { let dRuns = 0; let cctx: Ctx | null = null; const a = node([], null, { initial: 1 }); @@ -109,14 +109,26 @@ describe("C-4 mixed sync/async diamond (R-diamond, R-two-phase, R-first-run-gate ]); }); - collect(d); + const { msgs } = collect(d); // b settled synchronously (11); the async leg is deferred -> first-run gate holds d expect(dRuns).toBe(0); expect(d.cache).toBeUndefined(); - (cctx as Ctx).down([["DATA", 21]]); // async leg resolves + (cctx as Ctx).down([["DATA", 21]]); // async leg resolves -> first join (activation) expect(dRuns).toBe(1); // joined exactly once expect(d.cache).toBe(32); // 11 + 21 + + // R-dirty-before-data: a non-activation tier-3 emission is preceded by a synthesized + // DIRTY in the same wave (the join fn calls ctx.down([["DATA",...]]) only). Drive a + // second settle through both legs so we observe d past its first-run exemption. + msgs.length = 0; + a.down([["DATA", 2]]); // re-drives b (sync, 12) and c (async, deferred again) + expect(dRuns).toBe(1); // still gated on the async leg + (cctx as Ctx).down([["DATA", 30]]); // async leg re-resolves + expect(dRuns).toBe(2); + expect(d.cache).toBe(42); // 12 + 30 + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // DIRTY precedes DATA, glitch-free two-phase + expect(msgs.at(-1)).toEqual(["DATA", 42]); }); }); From 633bf87f8da685bde81e9f74a465d06b3754592b Mon Sep 17 00:00:00 2001 From: David Chen Date: Thu, 28 May 2026 22:29:47 -0700 Subject: [PATCH 005/175] refactor: update design-review and dev-dispatch SKILL documentation for clarity - Revised descriptions in both SKILL documents to reflect the clean-slate GraphReFly redesign. - Enhanced guidance on usage scenarios and clarified the workflow processes for design reviews and development dispatch. - Removed outdated references and streamlined the structure for better readability and understanding. --- .claude/skills/design-review/SKILL.md | 213 +++++++++----------------- .claude/skills/dev-dispatch/SKILL.md | 139 +++++++++-------- 2 files changed, 145 insertions(+), 207 deletions(-) diff --git a/.claude/skills/design-review/SKILL.md b/.claude/skills/design-review/SKILL.md index 4744f96e..5a783b32 100644 --- a/.claude/skills/design-review/SKILL.md +++ b/.claude/skills/design-review/SKILL.md @@ -1,26 +1,20 @@ --- name: design-review description: "Validate the design of a new primitive or API surface against the 5-question lens (Q5–Q9 from the per-unit review format). Use BEFORE coding (or right after a sketch lands) when adding a new public API / pattern factory / domain primitive. Triggers: 'design review', 'review the design', 'is this the right shape', 'before I implement'. Different from /qa — that finds bugs in landed code; this validates abstraction + long-term shape + reactive composability + alternatives." +disable-model-invocation: true argument-hint: "[ | | --diff] [optional context]" --- -You are executing the **design-review** workflow for **GraphReFly** (cross-language: TypeScript + Python). +You are executing the **design-review** workflow for the **clean-slate GraphReFly** redesign. -This skill applies the 5 design-review questions used in `archive/docs/SESSION-ai-harness-module-review.md` (Q5–Q9 of the 9-question per-unit format). The format originated for module-scope reviews; this skill makes it available for **single-symbol / single-file / single-diff** reviews. +This skill applies the 5 design-review questions (Q5–Q9 of the 9-question per-unit format) to a **single-symbol / single-file / single-diff** review. Use it BEFORE coding a new public API / sugar factory / operator / inspection surface — or right after a sketch lands, before tests. Different from `/qa` (finds bugs in landed code) and `/decision-guard` (recalls locked decisions); this validates abstraction + long-term shape + reactive composability + alternatives, and its output may become a new `D#` (architectural lock → user approval → append `decisions.jsonl`). -**When to use this skill:** +Clean-slate code lives in **`packages/ts/src/`** (`@graphrefly/ts`, D32). The language-neutral authority is **`~/src/graphrefly`** jsonl (branch `clean-slate`) — when this skill and that repo disagree, that repo wins (CLAUDE.md). -- Before implementing a new public API in `packages/pure-ts/src/patterns/` or `packages/pure-ts/src/extra/` -- Right after sketching a new factory / bundle / primitive, before tests are written -- When two slightly-different implementations exist and you need a principled pick -- When `/dev-dispatch` Phase 2 (architecture discussion) needs to go deeper than the default 4-question template +> **Stale-infra guard.** Do NOT cite the retired port-model: `packages/pure-ts/**` (frozen read-only reference only, D41), `docs/implementation-plan.md` / `optimizations.md` / `roadmap.md` / `test-guidance.md` / `docs-guidance.md`, `GRAPHREFLY-SPEC.md` / `COMPOSITION-GUIDE.md` (migrated to `spec/rules.jsonl` + `guide/guide.jsonl`, B7), `describe({format})` (D39: renderers are pure fns over the snapshot, NOT a `format` option), the `Impl`/facade/actor model. -**When NOT to use this skill:** - -- Bug fixes — use `/qa` -- Pure refactors that don't change the API surface — use `/qa` -- Implementation work that's already been approved at Phase 2 — proceed without this overhead -- Small additive changes (e.g. adding a JSDoc field, fixing a docstring) +**When to use:** before a new public symbol in `packages/ts/src/graph/` (sugar / operator / inspection) or a new substrate primitive in `packages/ts/src/{node,dispatcher,ctx,protocol,batch}/`; right after sketching a factory; when two implementations exist and you need a principled pick; when `/dev-dispatch` Phase 2 needs to go deeper than its default template. +**When NOT:** bug fixes / pure refactors → `/qa`; already-approved work → proceed; trivial additive changes (a JSDoc field, a docstring). User context: $ARGUMENTS @@ -28,191 +22,126 @@ User context: $ARGUMENTS ## Phase 0: Scope resolution -Determine the review target(s) from `$ARGUMENTS`: - -1. **`--diff` (or no args)** — review the new public symbols introduced in the current uncommitted diff. Run `git diff --name-only HEAD` and `git status --short` to enumerate. Filter to files in `packages/pure-ts/src/patterns/`, `packages/pure-ts/src/extra/`, `packages/pure-ts/src/core/`, `packages/pure-ts/src/graph/`, `packages/pure-ts/src/compat/`, or `packages/pure-ts/src/integrations/` that introduce new exports. -2. **``** — review the public symbols in that file (single-file scope). -3. **``** — locate the symbol with Grep, then review it (single-symbol scope). -4. **Multiple targets** — apply Q5–Q9 to each, then add Phase 2 cross-cutting synthesis. +Resolve the target(s) from `$ARGUMENTS`: -Read these in parallel before reviewing: +1. **`--diff` / no args** — review the new public symbols in the uncommitted diff. Enumerate via `git diff --name-only HEAD` + `git status --short`, filtered to `packages/ts/src/**` files that introduce new exports. +2. **``** — public symbols in that file. +3. **``** — Grep-locate, then review. +4. **Multiple targets** — apply Q5–Q9 to each, then add Phase 2 synthesis. -- `~/src/graphrefly/GRAPHREFLY-SPEC.md` § 5.8–5.12 (design invariants) -- `~/src/graphrefly/COMPOSITION-GUIDE.md` § 1, 2, 3, 5, 7, 8, 28, 32 (composition patterns) -- `docs/implementation-plan.md` — find the phase the target belongs to (Phase 11–16). The phase entry shows scope locks, design-session IDs (DS-#), and dependencies. If the target's design is locked in a phase, the design review validates against that lock; if not yet locked, the review's output may need to land as a new sub-phase entry or DS-# session. -- `docs/optimizations.md` "Active work items" (related architectural questions in flight, line-item state) -- `docs/roadmap.md` — vision context only (no longer the active sequencer; consult only for the strategic frame) -- The target file(s) themselves -- 1–2 closest existing primitives in the same directory (precedent) -- Optionally `~/src/callbag-recharge/` for analogous prior art (NOT spec authority) +Read in parallel before reviewing (clean-slate authority): -If the target imports anything from `packages/pure-ts/src/patterns/`, `~/src/graphrefly/COMPOSITION-GUIDE.md` is **mandatory** reading — composition primitives have non-obvious load-bearing patterns documented there. +- `~/src/graphrefly/spec/rules.jsonl` — the R-* rules your target touches (the design invariants). +- `~/src/graphrefly/decisions/decisions.jsonl` — the governing D# (or `/decision-guard` to recall the floor/values). +- `~/src/graphrefly/plan/phases.jsonl` — the CSP-* phase the target belongs to (locked vs open-design); if the target isn't sequenced yet, the review's output may need a new phase / backlog entry. +- `~/src/graphrefly/guide/guide.jsonl` (G-composition) — composition patterns (lazy activation, subscription order, SENTINEL/prevData guards, feedback cycles). +- `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) — the F-* constraints + the why behind locks. +- The target file(s) + 1–2 closest existing primitives in the same dir (precedent). +- **Frozen reference (D41):** `packages/pure-ts/**` + `~/src/callbag-recharge` for analogous prior art (operator behavior, edge cases, test structure) — NOT the authority. --- ## Phase 1: Per-target review (Q5–Q9) -For each target, produce a structured report covering all five questions. Be specific and quote file:line refs. Cap each answer at ~150 words; expand only when the topic genuinely needs it. +For each target, produce a structured report covering all five questions. Be specific, quote `file:line`. Cap each answer ~150 words. ### Q5 — Is this the right abstraction? Could it be more generic? -Probe: - -- **Layer placement.** Does it belong in `core/` (protocol primitive), `extra/` (operator/source), `patterns/` (Phase 4+ domain factory), or `compat/` (framework adapter)? Mismatched layer is the most common drift signal. -- **Decomposition.** Could the body be split into 2+ smaller primitives that compose into this one? (Smaller pieces are more likely to be reused.) -- **Generalization.** Is there a 2+ similar primitive nearby that hints at a shared abstraction? Could `T = number` be `T = unknown` without losing safety? Could the config object be split into smaller bundles? -- **Naming.** Does the name describe **what it returns / produces**, or **what it does internally**? The former is composable; the latter rots. - -Output: - -> **Layer:** {core/extra/patterns/compat/integrations} — {fits / mismatches because …} -> **Decomposition:** {already minimal / could split into A + B / over-decomposed} -> **Generalization:** {none surfaced / could collapse with X / over-general} -> **Naming:** {clear / ambiguous because …} +- **Layer placement.** Substrate (`node`/`dispatcher`/`ctx`/`protocol`/`batch` — a protocol primitive, must stay thin per R-node-thin) vs graph-layer (`graph/` — sugar / operator / inspection, per-language per D6/D24)? Mismatched layer is the most common drift signal. A new **verb** is a constitutional change (8-verb closed set, D4) — almost certainly the target is sugar, not a verb. +- **Decomposition.** Could the body split into 2+ smaller primitives that compose? (F-NO-WEDGE-CUT: each must serve ≥2 segments.) +- **Generalization.** A 2+ similar primitive nearby hinting at a shared abstraction? Could `T = number` be `T = unknown` without losing safety? +- **Naming.** Does the name describe what it **returns/produces** (composable) or what it does internally (rots)? D6 real factory names show in `describe`. -### Q6 — Is this the right long-term solution? What are the caveats? Maintenance burden? +> **Layer / Decomposition / Generalization / Naming:** … -Probe: +### Q6 — Right long-term solution? Caveats? Maintenance burden? -- **6-month lens.** What changes in the surrounding codebase would force this to evolve? Spec changes? Phase 5 work? PY parity? -- **Special cases / hidden invariants.** Anything that the type system can't express but the implementation depends on (subscribe order, init-time-vs-runtime, sync-vs-async strategies, stable references, etc.)? List each one explicitly — these become bug-class footguns. -- **Constraint locks.** Does the chosen shape close off future extensions? (E.g. positional-arg constructor → can't add options later without breaking; hard-coded enum → can't extend with custom variants.) -- **Documentation debt.** Is the contract knowable from JSDoc alone, or does it require reading the body? Hidden contracts age badly. +- **6-month lens.** What forces this to evolve — spec/conformance changes, rust/py arm parity (D24), F-PERF budget? +- **Hidden invariants** the type system can't express — list each as `INVARIANT: …` (subscribe order before kick; first-run gate; SENTINEL = `prevData === undefined`; sync-vs-async strategy; stable refs; equals identity). +- **Constraint locks** (positional args can't grow options; hardcoded enum can't extend). +- **Doc debt** (contract knowable from JSDoc, or only from the body?). -Output: +> **6-month risk / Hidden invariants / Constraint locks / Doc debt:** … -> **6-month risk:** {low / medium / high — because …} -> **Hidden invariants:** {bullet list, each as `INVARIANT: …`} -> **Constraint locks:** {none / list each} -> **Doc debt:** {complete / requires reading body / undocumented} +### Q7 — Can we simplify it? Reactive, composable, explainable? -### Q7 — Can we simplify it? Make it reactive, composable, explainable? +The **explainability check** — a primitive's reactive shape is only as good as its `describe()` snapshot (D39). -This is the **explainability check** — borrowed from the pagerduty-demo + AI/harness-module-review insight. **A primitive's reactive shape is only as good as its `describe()` output.** +- **Wire a minimal composition** (≥2 sources → target → ≥1 sink). Predict `describe()` — a flat JSON-serializable snapshot; renderers (pretty/mermaid/d2) are pure fns over it (D39), NOT a `describe({format})` option. If you can't predict the output, the topology is too imperative. +- **Island check.** A node with zero in-edges AND zero out-edges (not an entry/exit) is a smell. +- **Imperative escape paths.** Search for: emit/set/callback wiring that bypasses the graph (R-no-imperative); `.cache` reads inside reactive fn bodies (R-data-not-peek — data moves via messages); raw `Promise`/`setTimeout`/`queueMicrotask` outside a source/pool (R-no-raw-async / F-SYNC-CORE); hardcoded `type === "DATA"` instead of `messageTier` (R-tier); an inline fn bypassing the dispatcher (R-dispatch-all). +- **SENTINEL / prevData guards.** Never-emitted detection via `ctx.prevData[i] === undefined` (the canonical detector); fix eager-placeholder upstreams rather than bolting on companions. +- **Feedback cycles.** A fn that re-drives its own dep mid-wave is a wave-level ERROR (D37/R-reentrancy), not iteration; legit accumulation = `ctx.state` (scan), not a topological cycle. -Probe: - -- **Wire a minimal composition.** Imagine `≥2 upstream sources → target → ≥1 downstream sink`. What does `graph.describe({ format: "ascii" })` show? If you can't predict the output, the topology is too imperative. -- **Island check.** Does any node in the target have zero in-edges AND zero out-edges (and isn't an entry/exit)? Smell. -- **Imperative escape paths.** Search for: `.emit()` / `.set()` / `.publish()` calls inside fn bodies (vs effect bodies); `.cache` reads inside reactive fn bodies (COMPOSITION-GUIDE §28); raw `Promise` / `setTimeout` / `queueMicrotask` (spec §5.10); event-emitter or callback wiring that bypasses the graph; hardcoded message-type checks (`type === DATA`) instead of `messageTier` (spec §5.11). -- **Closure-mirror correctness.** If the target captures upstream values via closure subscribe, is the seed correct (COMPOSITION-GUIDE §28)? Is the subscribe ordered before any kick (`feedback_subscribe_before_kick.md`)? -- **Feedback cycles.** If the target writes back to a node it reads, is the cycle broken cleanly (snapshot-on-settle, batch boundaries, §32 state-mirror)? - -Output: - -> **Topology check:** {clean — describe walks cleanly | islands: X / Y / Z | imperative leaks: …} -> **Imperative escape paths:** {none | list each} -> **Closure / cache reads:** {none | each one's justification} -> **Feedback cycles:** {none | broken by … / NOT broken — concern} -> **Simplification opportunities:** {none | bullet list — each with the cost it would impose} +> **Topology / Imperative leaks / cache reads / Feedback cycles / Simplifications:** … ### Q8 — Alternative implementations (A / B / C) -Sketch **at least 2** named alternatives. For each, give: - -- **Shape** — 1–3 lines of pseudo-signature -- **Pros** — 2–4 bullets -- **Cons** — 2–4 bullets -- **Precedent** — does this shape exist in `callbag-recharge`, RxJS, or another reactive lib? Cite if so. +Sketch **≥2** named alternatives. For each: **Shape** (1–3 line pseudo-sig), **Pros** (2–4), **Cons** (2–4), **Precedent** — does the shape exist in the frozen `packages/pure-ts/**` reference (D41), `callbag-recharge`, or RxJS? Cite if so. Don't pick a winner yet — that's Q9. -Don't pick a winner yet — that's Q9. - -Output: - -> **A. {name}** -> ```ts -> // sketch -> ``` -> Pros: … -> Cons: … -> Precedent: … -> -> **B. {name}** -> … +> **A. {name}** — sketch / Pros / Cons / Precedent +> **B. {name}** — … ### Q9 — Recommendation + coverage check -Pick the recommended alternative from Q8. Then build a coverage matrix: - -| Concern from Q5–Q8 | Recommended alt covers it? | -|---|---| -| {abstraction concern} | yes / partially / no — because … | -| {invariant from Q6} | … | -| {feedback cycle from Q7} | … | -| {alternative B's pro that we'd lose} | … | - -If any row is **partially** or **no**, name the residual risk explicitly. Either: - -- The risk is acceptable (justify why) -- Add a follow-up to `docs/optimizations.md` "Active work items" tracking the gap -- Pick a different alternative +Pick the recommended alternative; build a coverage matrix (each Q5–Q8 concern → recommended alt covers it? yes / partially / no — because …). For any `partially`/`no`, name the residual risk: accept it (justify), add a `backlog.jsonl` follow-up, or pick a different alternative. End with: -End with a one-paragraph summary the user can act on: +> **Recommendation:** {alt}, because {2–3 reasons grounded in Q5–Q8}. +> **Residual risks:** {none, OR 1–2}. +> **Implementation guidance:** {next step — usually a draft `D#` for approval, or a sub-decision the user must answer first}. -> **Recommendation:** {alternative}, because {2-3 reasons grounded in Q5–Q8}. -> **Residual risks:** {none, OR list 1–2}. -> **Implementation guidance:** {1-3 sentences on what to do next — usually approval to proceed, sometimes a sub-decision the user must answer first}. +If the design is an architectural lock, draft the `D#` (`{id, layer, date, question, decision, rationale, supersedes, status}`) for the user to approve BEFORE append — do NOT auto-lock (no-autonomous-decisions). A wave-protocol behavior change routes to `/spec-amend` (spec-first), not a direct edit. --- ## Phase 2: Cross-cutting synthesis (multi-target only) -Apply only if reviewing multiple targets in the same pass (e.g. `--diff` mode with 3 new factories). +Apply only when reviewing multiple targets in one pass: -Cross-cutting checks: +- **Naming consistency** (`extract` vs `select` vs `pick` for the same role = drift). +- **Argument-shape consistency** (options-bag vs positional applied the same way). +- **Composition direction** (do the targets' input/output shapes line up to compose?). +- **Repeated patterns** (the same SENTINEL-gate / subscribe-order / batch-on-write in 2+ places → a shared helper candidate). -- **Naming consistency.** Are similar concepts named consistently across targets? (`extract` vs `select` vs `pick` for the same role is drift.) -- **Argument shape consistency.** Is the options-bag vs positional-args split applied the same way? -- **Composition direction.** Are the targets meant to compose? If so, do their input/output shapes line up cleanly? -- **Repeated patterns.** Did the same closure-mirror / SENTINEL-gate / batch-on-write pattern appear in 2+ places? That's a candidate for a shared helper. - -Output a numbered list of cross-cutting findings, each with: - -- The pattern/inconsistency -- Where it appears (file:line × N) -- A proposed unifying shape +Output a numbered list, each finding with the pattern, where it appears (file:line × N), and a proposed unifying shape. --- ## Phase 3: Decisions log -For any **architectural** question that doesn't have a clear answer in the synthesis (e.g. "should we generalize X across both Y and Z?"), append it to `docs/optimizations.md` under "Active work items" using the standard shape: - -``` -- **{Title} ({date}, design-review).** {Question}. Options: {A / B / C}. Tradeoff: {…}. Blocked on: {concrete consumer / spec clarification / further design pass}. -``` - -Resolved decisions move to `archive/optimizations/resolved-decisions.jsonl` per `docs/docs-guidance.md` § "Optimization decision log" (only after the user picks one). When a design decision lands as part of a phase that fully completes, the matching `docs/implementation-plan.md` phase body should also be archived to `archive/roadmap/phase--*.jsonl` per `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/implementation-plan.md`" — flag this in the recommendation if the design under review closes out a phase. +- **Architectural lock** (clear) → draft a `D#` for `~/src/graphrefly/decisions/decisions.jsonl`; append only after user approval, then update the DS-1 `locks` in `sessions/sessions.jsonl` and run `node ~/src/graphrefly/dashboard/build.mjs --check`. +- **Deferred / no clear answer** → append to `~/src/graphrefly/plan/backlog.jsonl` (B# + concrete trigger). +- **Recurring anti-pattern** → `~/src/graphrefly/plan/antipatterns.jsonl` (+ a `feedback_*` memory if generalizable). +- **Protocol-behavior change** surfaced → route to `/spec-amend` (spec-first), not a direct code change. --- ## Output discipline - Be concrete. Quote `file:line` refs. -- Don't write "this looks good" — if it does, say WHY (which Q5–Q9 dimensions clear). +- Don't write "this looks good" — say WHICH Q5–Q9 dimensions clear and why. - Don't pad. If Q6 has no caveats, write `**Hidden invariants:** none surfaced.` and move on. -- Don't second-guess the user's stated intent. Q8 alternatives are options to compare; Q5–Q7 probe the recommended shape. -- Output should be skim-readable: headers per question, bullets within. +- Don't second-guess the user's stated intent — Q8 alternatives are options to compare; Q5–Q7 probe the recommended shape. +- Skim-readable: headers per question, bullets within. --- ## Authority hierarchy -1. **`~/src/graphrefly/GRAPHREFLY-SPEC.md`** — protocol contract -2. **`~/src/graphrefly/COMPOSITION-GUIDE.md`** — composition patterns -3. **`docs/implementation-plan.md`** — pre-1.0 phase locks (canonical sequencer); the matching phase entry holds locked design decisions for active work -4. **`docs/test-guidance.md`** — testability shape -5. Existing patterns in `packages/pure-ts/src/` — only when the above are silent -6. **`docs/roadmap.md`** — vision context only; consult for strategic frame, not phase scope +1. `~/src/graphrefly/spec/rules.jsonl` — the protocol 宪法. +2. `~/src/graphrefly/decisions/decisions.jsonl` (+ DS-1 narrative) — locked decisions + F-* floor + durable values. +3. `~/src/graphrefly/plan/phases.jsonl` — the CSP-* sequencer (phase locks). +4. `~/src/graphrefly/guide/guide.jsonl` (G-test / G-composition) — testability + composition shape. +5. Existing patterns in `packages/ts/src/` — only when the above are silent. -If a finding seems to conflict with a higher-authority document, surface it explicitly — DO NOT silently override. +If a finding conflicts with a higher-authority doc, surface it explicitly — DO NOT silently override (no-autonomous-decisions). --- ## What to do AFTER this skill completes -- If recommendation is locked: invoke `/dev-dispatch` (or `/dev-dispatch --light` for small changes) with the locked design. -- If decisions are deferred: leave them in `docs/optimizations.md` and move on. -- If the design needs more thought: HALT, summarize, and let the user think. +- Lock approved → `/dev-dispatch` (or `--light`) with the locked design; a protocol-behavior change goes through `/spec-amend` first. +- Decisions deferred → leave them in `backlog.jsonl` and move on. +- Needs more thought → HALT, summarize, let the user think. -This skill produces a report; it does NOT modify implementation files (only `docs/optimizations.md` if architectural decisions are logged). +This skill produces a report; it modifies no implementation files (it only appends to `~/src/graphrefly` jsonl after explicit user approval of a `D#`). diff --git a/.claude/skills/dev-dispatch/SKILL.md b/.claude/skills/dev-dispatch/SKILL.md index 035e34b7..4288cccf 100644 --- a/.claude/skills/dev-dispatch/SKILL.md +++ b/.claude/skills/dev-dispatch/SKILL.md @@ -4,91 +4,101 @@ description: "Implement feature/fix with planning and self-test. Use when user s argument-hint: "[--light] [task description or context]" --- -You are executing the **dev-dispatch** workflow for **GraphReFly** (cross-language: TypeScript + Python). +You are executing the **dev-dispatch** workflow for the **clean-slate GraphReFly** redesign. -Operational docs, roadmap, optimizations, and skills all live in **graphrefly-ts** (this repo). Implementation may target `graphrefly-ts`, `graphrefly-py` (`~/src/graphrefly-py`), or both. +This repo is **`@graphrefly/ts`** — the self-contained TypeScript implementation (D32). Clean-slate code lands in **`packages/ts/src/`**. The language-neutral authority (spec / decisions / plan / conformance / formal) lives in **`~/src/graphrefly`** (branch `clean-slate`) as jsonl — when this skill and that repo disagree, **that repo wins** (CLAUDE.md). Sibling impls: `@graphrefly/rust` (`~/src/graphrefly-rs`), `@graphrefly/py` (`~/src/graphrefly-py`) — each self-contained; cross-language = wire bridge, never in-process (D32). + +> **Stale-infra guard.** Do NOT reach for the retired port-model surfaces: `packages/pure-ts/**` (frozen read-only reference only, D41), `docs/implementation-plan.md`, `docs/optimizations.md`, `docs/roadmap.md`, `docs/test-guidance.md`, `docs/docs-guidance.md`, `GRAPHREFLY-SPEC.md`/`COMPOSITION-GUIDE.md` (migrated to `spec/rules.jsonl` + `guide/guide.jsonl`, B7), the `Impl`/facade/actor model / 3-digit D### port decisions. The clean-slate authority is the jsonl below. The user's task/context is: $ARGUMENTS ### Mode detection +If `$ARGUMENTS` contains `--light`, this is **light mode**. Otherwise **full mode**. Differences are noted inline per phase. -If `$ARGUMENTS` contains `--light`, this is **light mode**. Otherwise, this is **full mode**. Differences are noted inline per phase. +### Workflow floor (non-negotiable) +- **decision-first**: any architectural lock needs a `D#` in `~/src/graphrefly/decisions/decisions.jsonl` BEFORE code (`/design-review` → user approval → append). Decisions locked ≠ implementation approved — wait for an explicit "implement". +- **spec-first** (F-NO-IMPL-DEFINED): any wave-protocol behavior change amends `spec/rules.jsonl` + `formal/*.tla` + `spec/conformance.jsonl` FIRST (`/spec-amend`), THEN code. Operators/sugar/inspection are per-language (D6/D24) — NOT spec, skip spec-amend. +- **no autonomous decisions**: surface spec↔code conflicts; don't silently pick. File-by-file review for multi-file rewrites. +- **verify premise**: design tables lag code — grep the named symbols + check landed markers (`plan/phases.jsonl` status/notes) before designing new surface; a stale premise is a HALT. +- **consistency gate**: after touching any `~/src/graphrefly` jsonl, run `node ~/src/graphrefly/dashboard/build.mjs --check` (non-zero on broken links / orphans). --- ## Phase 1: Context & Planning -Load context and plan the implementation in a single pass. **Parallelize all reads.** - -Read in parallel: -- `~/src/graphrefly/GRAPHREFLY-SPEC.md` — behavior authority; deep-read sections relevant to the task -- `~/src/graphrefly/COMPOSITION-GUIDE.md` — **composition patterns and insights** (read when building Phase 4+ factories that compose primitives — covers lazy activation, subscription ordering, null guards, Versioned navigation, factory wiring order) -- `docs/implementation-plan.md` — **CANONICAL pre-1.0 sequencer**. Find the phase this task belongs to (Phase 11 cleanup / Phase 12 consolidation / Phase 13 multi-agent / Phase 14 changesets / Phase 14.5 residuals / Phase 15 eval / Phase 16 launch). The phase entry tells you what's locked, what's still open-design (DS-#), and what cross-references back to `optimizations.md` for line-item state. Read this FIRST so you know whether you're picking up a NOW item or one tagged WAIT/POST-1.0. -- `docs/optimizations.md` — **line-item state for individual deferred carries**, anti-patterns, and **deferred follow-ups** (read when touching protocol, batch, node lifecycle, or parity, OR when the implementation-plan phase entry references an optimization-id). Resolved decisions are archived in `archive/optimizations/*.jsonl` — search there for historical context (see `docs/docs-guidance.md` § "Optimization decision log") -- `docs/test-guidance.md` — checklist for the relevant layer (core protocol, node, graph, extra) -- `docs/roadmap.md` — **vision / wave context only** (no longer the sequencer per 2026-04-30 migration — implementation-plan.md is canonical). Read for the strategic frame on a feature, not to find what's next. -- Any files the user referenced in $ARGUMENTS -- Relevant source files in the area you'll modify (TS: `packages/pure-ts/src/`, PY: `~/src/graphrefly-py/src/graphrefly/`) -- Existing tests for the area (TS: `packages/pure-ts/src/__tests__/`, PY: `~/src/graphrefly-py/tests/`) - -**Mandatory for patterns/ work:** If the task touches any file in `packages/pure-ts/src/patterns/` or `packages/pure-ts/src/compat/`, reading `~/src/graphrefly/COMPOSITION-GUIDE.md` is **mandatory**, not optional. The harness, orchestration, messaging, and all Phase 4+ code are composed factories — modifying their tests or implementation requires understanding composition patterns (lazy activation, subscription ordering, feedback cycles, SENTINEL gate). - -**Roadmap §2.3 (sources & sinks):** implement as thin wrappers over the **`node` primitive** (`node`, `producer`, `derived`, `effect`) and the message protocol — no parallel source/sink protocol outside `node`. - -While planning, explicitly validate proposed changes against these invariants (from the spec and roadmap): -- **Tier placement (TS)** — every new TS public symbol picks a tier: **universal** (default, browser + Node safe), **node-only** (`/node` subpath, may import `node:*`), or **browser-only** (`/browser` subpath, may use DOM globals). See `docs/docs-guidance.md` § "Browser / Node / Universal split". If the symbol imports `node:fs`, `node:path`, `node:crypto`, `node:sqlite`, `node:child_process`, etc., it belongs in `/node` — NOT in the universal barrel. If it imports `window` / `document` / `indexedDB` / DOM types, it belongs in `/browser`. Adding a new subpath means updating `packages/pure-ts/tsup.config.ts` `ENTRY_POINTS` (+ `nodeOnlyEntries` for node-only) AND `packages/pure-ts/package.json` `exports`. -- **Control flows through the graph** — lifecycle and coordination use messages and topology, not imperative bypasses around the graph (spec §5.1). -- **Messages are always** `[[Type, Data?], ...]` — no single-message shorthand. -- **DIRTY before DATA/RESOLVED** in the same logical update where two-phase push applies; **batch** defers DATA, not DIRTY. -- **Unknown message types forward** — do not swallow unrecognized tuples. -- Prefer **composition (nodes + edges)** over monolithic configuration objects. -- For **diamond** topologies, recomputation happens once per upstream change after all deps settle. -- **No polling** — never poll node values on a timer or busy-wait. Use reactive sources (`fromTimer`/`from_timer`, `fromCron`/`from_cron`) instead (spec §5.8). -- **No imperative triggers** — no event emitters, callbacks, or `setTimeout`/`threading.Timer` + `set()` workarounds. All coordination uses reactive `NodeInput` signals (spec §5.9). -- **No raw async primitives** — TS: no bare `Promise`, `queueMicrotask`, `setTimeout`, `process.nextTick`. PY: no bare `asyncio.ensure_future`, `asyncio.create_task`, `threading.Timer`, or raw coroutines. Async belongs in sources and the runner layer (spec §5.10). -- **Central timer and `messageTier`/`message_tier`** — TS: use `core/clock.ts`. PY: use `core/clock.py`. Never hardcode type checks (spec §5.11). -- **Phase 4+ APIs must be developer-friendly** — sensible defaults, minimal boilerplate, clear errors. Protocol internals never surface in primary APIs (spec §5.12). -- **PY: Thread safety** — design for GIL and free-threaded Python where core APIs are documented as thread-safe. Per-subgraph `RLock` (see roadmap Phase 0.4). -- **PY: No `async def` in public APIs** — all public functions return `Node[T]`, `Graph`, `None`, or a plain synchronous value. +Load context and plan in a single pass. **Parallelize all reads.** + +Read in parallel (clean-slate authority): +- `~/src/graphrefly/CLAUDE.md` — the single-source authority index (read FIRST). +- `~/src/graphrefly/spec/rules.jsonl` — the protocol 宪法 (R-* rules); deep-read the rules your change touches. +- `~/src/graphrefly/decisions/decisions.jsonl` — the unified D# log (or invoke `/decision-guard` to recall the governing D#/values/floor). +- `~/src/graphrefly/plan/phases.jsonl` — the CSP-* sequencer: find the phase this task belongs to, its `status` (done/impl/design), deps, and note. Read this FIRST among the plan files so you know whether you're on a ready phase or one still gated. +- `~/src/graphrefly/plan/backlog.jsonl` + `plan/antipatterns.jsonl` — deferred carries (B#) with triggers; anti-patterns to flag against. +- `~/src/graphrefly/spec/conformance.jsonl` — the behavioral scenarios (C-*) your change must keep green; check the `runtimes` status for the arm you target. +- `~/src/graphrefly/guide/guide.jsonl` — composition / test / docs / contribute guidance (G-composition / G-test / G-docs / G-contribute). +- `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) — the L0–L6 design narrative + F-* constraints, when you need the why behind a lock. +- Any files the user referenced in $ARGUMENTS. +- The clean-slate source you'll modify: substrate = `packages/ts/src/{node,dispatcher,ctx,protocol,batch}/`; graph-layer = `packages/ts/src/graph/` (Graph + 8-verb sugar + operators + inspection describe/observe/profile). +- Existing tests: `packages/ts/src/__tests__/`. + +**Frozen reference (D41):** `packages/pure-ts/**` and `~/src/callbag-recharge` are READ-ONLY prior art for analogous operator behavior / edge cases / test structure during a re-derive (D40 Catalog-first). Map concepts to the clean-slate substrate (`node`/`ctx.down`/`ctx.depRecords`/`Graph`, D39 `describe`/`observe`) — do NOT 1:1 port; the old substrate API and semantics differ. They are NOT the behavior authority — `spec/rules.jsonl` is. + +While planning, validate proposed changes against the clean-slate floor (cite the rule/D#): +- **Sacred (L0.7):** topology declarative/serializable/inspectable · wave protocol is a public spec · wave protocol impl is **sync** · all fn go through the dispatcher. +- **8 verbs, closed (D4):** `node`/`graph`/`batch`/`state` + `producer`/`derived`/`effect`/`mount`. **Operators are `node` sugar (D6), not verbs** — per-language, never in parity (D24); real factory names show in `describe`. Adding a verb is a constitutional change. +- **Messages** `[[Type, Data?], ...]`; one array to `ctx.down`/`ctx.up` = one wave (R-msg-format). 10-type closed set, no user-defined types (R-msg-closed-set). +- **DIRTY before DATA/RESOLVED** in the same wave (R-dirty-before-data); two-phase glitch-free diamond (R-two-phase); a diamond/fan-in node recomputes exactly once after all changed deps settle (R-diamond). batch defers tier-≥3, not DIRTY. +- **`ctx.up` is control-tier only** (DIRTY/PAUSE/RESUME/INVALIDATE/TEARDOWN); DATA/RESOLVED/COMPLETE/ERROR are down-only (R-ctx-up, D8). A handle is pure data, no methods (D7). +- **No polling** (R-no-polling); **no imperative triggers** (R-no-imperative — reactive `ctx.up`/signals, not emitters/callbacks/timers+set; remove imperative paths when no caller depends); **no raw async** in the sync core (R-no-raw-async / F-SYNC-CORE — async lives only in sources / the pool / the wire bridge). +- **All fn through the dispatcher** (R-dispatch-all / F-DISPATCH-ALL — no inline-fn bypass). `dispatcher.invoke` is sync void (R-sync-core). +- **Data moves via messages** (R-data-not-peek — never peek a dep's `.cache` to seed compute; `.cache` is a read-only accessor for external consumers). SENTINEL = absence-of-DATA (R-sentinel); the canonical never-emitted detector is `ctx.prevData[i] === undefined`. +- **messageTier is a compile-time const table** (D18/D34/R-tier); the clock is **graph-local** (no global singleton, D26/R-clock); `onMessage`/`onSubscribe` are substrate-fixed, not user-replaceable (D19). +- **Primary-API clean** (R-primary-api-clean): protocol internals (DIRTY/RESOLVED/bitmask) never surface in value-level sugar (derived/effect/operator); ctx-level (node/producer) intentionally exposes tier as a power surface (DR-1). Sugar value-fn → ctx-fn wrapping happens in the graph layer (D27); a value-level `throw` becomes `[[ERROR,e]]` down (D30). +- **graph = single-thread causal/concurrency domain (D22 / R-graph-domain):** parallelism via pool callback or multi-graph + wire bridge; rewire is intra-graph only. +- **`ctx.state`** = per-node private cross-wave state (R-ctx-state); shared/observable state must be an explicit node + dep, not `ctx.state` (D23). A synchronous feedback cycle (a fn re-driving its own dep mid-wave) is a wave-level ERROR (D37/R-reentrancy), not iteration. +- **F-NO-WEDGE-CUT:** every primitive serves ≥2 segments (no LLM-only or single-segment wedge; F-NO-LLM-ONLY). **F-PERF:** budget every abstraction (thin node, default-off inspection). + +**Targeting a sibling (py/rust):** if the task targets `@graphrefly/py` (`~/src/graphrefly-py`) or `@graphrefly/rust` (`~/src/graphrefly-rs`), read that package's local layout + its conformance arm status in `spec/conformance.jsonl`. The cross-language contract is **behavioral conformance (D24)**, not symbol parity. PY public APIs are synchronous (return `Node[T]`/`Graph`/value, no `async def`); async lives at the source/pool boundary only (F-SYNC-CORE). Do NOT start implementing yet. -**Optional context:** The predecessor codebase **callbag-recharge** at `~/src/callbag-recharge` has mature patterns, tests, and docs. Use it for **analogous** operator behavior, edge cases, and test structure — then map concepts to GraphReFly (unified message tuples, `node`, `Graph`, `describe`/`observe`). It is not the behavior spec; `~/src/graphrefly/GRAPHREFLY-SPEC.md` is. - --- ## Phase 2: Architecture Discussion ### Full mode — HALT -**HALT and report to the user before implementing.** Present: +**HALT and report before implementing.** Present: -1. **Architecture assumptions** — how this fits into `packages/pure-ts/src/core/`, `packages/pure-ts/src/graph/`, `packages/pure-ts/src/extra/` -2. **New patterns** — any new patterns not yet in this repo -3. **Options considered** — alternatives with pros/cons -4. **Recommendation** — preferred approach and why +1. **Architecture assumptions** — how this fits the substrate (`node`/`dispatcher`/`ctx`/`protocol`/`batch`) vs graph-layer (`graph/`) split. +2. **New patterns** — any not yet in `packages/ts/src/`. +3. **Options considered** — alternatives with pros/cons. +4. **Recommendation** — preferred approach + why. Prioritize (in order): -1. **Correctness** — matches `~/src/graphrefly/GRAPHREFLY-SPEC.md` and protocol invariants -2. **Completeness** — edge cases (errors, completion, reconnect, diamonds) -3. **Consistency** — matches patterns already in the target repo -4. **Simplicity** — minimal solution -5. **Thread safety** (PY) — where concurrent `get()` / propagation applies +1. **Correctness** — matches `~/src/graphrefly/spec/rules.jsonl` + the floor. +2. **Completeness** — edge cases (errors, COMPLETE, reconnect/reactivate, diamonds, SENTINEL gate, PAUSE lockset). +3. **Consistency** — matches patterns already in `packages/ts/src/`. +4. **Simplicity** — minimal solution. -Do NOT consider backward compatibility at this early stage (pre-1.0). +No backward compatibility (pre-1.0). -**Cross-language decision log:** If Phase 1–2 surface an **architectural or product-level** question (protocol semantics, batch/node invariants, parity between ports, or anything that needs a spec/product call), **jot it down** in **`docs/optimizations.md`** under **"Active work items"** (this repo is the single source of truth for both TS and PY). When the decision is **resolved**, move it to `archive/optimizations/resolved-decisions.jsonl` per `docs/docs-guidance.md` § "Optimization decision log". +**Escalation routing** (don't silently pick — no-autonomous-decisions): +- Architectural lock → `/design-review` → user approval → append a `D#` to `decisions.jsonl`. +- Wave-protocol behavior change → `/spec-amend` (spec-first: rules + TLA+ + conformance, THEN code). +- Cross-runtime concern → `/conformance` (behavioral scenario, not structural diff). +- Deferred/open question with no answer yet → append to `~/src/graphrefly/plan/backlog.jsonl` (B# + trigger); a recurring anti-pattern → `plan/antipatterns.jsonl` (+ a `feedback_*` memory if generalizable). **Wait for user approval before proceeding.** ### Light mode — Skip unless escalation needed Proceed directly to Phase 3 **unless** Phase 1 reveals any of these: -- Changes to **message protocol**, **node** semantics, or **core** push/pull behavior -- New patterns not present anywhere in the codebase -- Multiple viable approaches with non-obvious trade-offs +- A change to **wave-protocol behavior** (tiers, wave semantics, diamond/equals/SENTINEL, batch, push-on-subscribe, ctx.up/down contract) → spec-first, escalate. +- A new architectural lock with no governing `D#`. +- Multiple viable approaches with non-obvious trade-offs. -If any apply, escalate: HALT and present findings as in full mode. +If any apply: HALT and present findings as in full mode. --- @@ -96,19 +106,18 @@ If any apply, escalate: HALT and present findings as in full mode. After user approves (full mode) or after Phase 1 (light mode, no escalation): -1. Implement the changes - - Treat `~/src/graphrefly/GRAPHREFLY-SPEC.md` as non-negotiable for behavior - - If existing code drifts from the spec, align toward the spec as part of the change -2. Create tests following `docs/test-guidance.md`: - - Put tests in the most specific existing file under `packages/pure-ts/src/__tests__/` (or colocated `*.test.ts` per project convention) - - Use **`Graph.observe()`** / **`graph.observe()`** for live message assertions when the Graph API exists; until then, test at the **node** and **message** level per test-guidance +1. Implement the changes. + - Treat `~/src/graphrefly/spec/rules.jsonl` as non-negotiable for behavior; if code drifts from a rule, align to the rule — or surface the conflict, don't silently pick. + - Cite the governing R-id / D# in test expectations. +2. Create tests (per `guide/guide.jsonl` G-test — unit / property / conformance layering): + - Put tests in the most specific existing file under `packages/ts/src/__tests__/`. + - Use `graph.observe()` for live message assertions; assert at the node + message level otherwise. A behavioral-protocol change ALSO needs a `spec/conformance.jsonl` scenario (`/conformance`) before its rule flips `draft → active`. 3. Run checks: - - **TS:** `pnpm test` — and when the change touches `packages/pure-ts/src/extra/` or `packages/pure-ts/src/patterns//`, also `pnpm run build` so the post-build `assertBrowserSafeBundles` guardrail catches any Node-builtin that leaked into a universal entry. - - **PY:** `cd ~/src/graphrefly-py && uv run pytest && uv run ruff check src/ tests/ && uv run mypy src/` -4. Fix any failures - -If implementation leaves an **open architectural decision** (deferred behavior, parity caveat, or “needs spec” item), add it to **`docs/optimizations.md`** under “Active work items” (this repo is the single source of truth). When resolved, archive to `archive/optimizations/resolved-decisions.jsonl` per `docs/docs-guidance.md`. + - **TS:** `pnpm --filter @graphrefly/ts test` (vitest) + `pnpm run lint` (biome + layer/typecheck gates) + `pnpm run build` (tsup) as relevant. + - **PY (if targeted):** the `@graphrefly/py` package's own test/lint/type gates in `~/src/graphrefly-py`. + - **jsonl touched:** `node ~/src/graphrefly/dashboard/build.mjs --check` (consistency gate). +4. Fix any failures. -If implementation **closes the last in-flight item from a Phase 11–16 sub-section** in `docs/implementation-plan.md`, mark the sub-section ✅ inline. If it closes the **last in-flight item in a whole Phase**, also archive the phase body to the matching `archive/roadmap/phase--*.jsonl` and replace the phase body with a 2–4-line summary + archive pointer per `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/implementation-plan.md`". Single residual follow-ups move to `docs/optimizations.md` with a back-link to the archived phase id. Same convention applies to fully-completed waves / sections in `docs/roadmap.md`. +If implementation leaves an **open architectural decision** (deferred behavior, parity caveat, "needs spec" item), append it to `~/src/graphrefly/plan/backlog.jsonl` (B# + trigger) — NOT a docs file. If it **lands or advances a CSP-* phase**, update that phase's `status`/`note` in `~/src/graphrefly/plan/phases.jsonl`, flip any conformance-backed `draft` rule to `active` once its scenario is green per arm, then run the consistency gate. When done, briefly list files changed and new exports added. Then suggest running `/qa` for adversarial review and final checks. From 4c2677385c3fce8935646c07cdaeb1b3b9db2c9a Mon Sep 17 00:00:00 2001 From: David Chen Date: Thu, 28 May 2026 23:11:04 -0700 Subject: [PATCH 006/175] feat(tests): add comprehensive intra-graph rewire tests and enhance node functionality - Introduced new test suite for intra-graph runtime rewire scenarios, covering various Q-semantics including addDep, removeDep, and setDeps. - Implemented detailed test cases to validate cache preservation, idempotency, and rejection of invalid rewiring operations. - Enhanced the Node class to support surgical rewire operations, ensuring efficient dependency management and cycle prevention. - Updated the internal structure to maintain state across dependency changes, improving performance for high fan-in scenarios. --- packages/ts/src/__tests__/conformance.test.ts | 48 ++++ packages/ts/src/__tests__/rewire.test.ts | 192 +++++++++++++++ packages/ts/src/node/node.ts | 223 +++++++++++++++++- 3 files changed, 453 insertions(+), 10 deletions(-) create mode 100644 packages/ts/src/__tests__/rewire.test.ts diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index dcf143ab..382161a1 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -224,3 +224,51 @@ describe("C-7 upstream control at a depless source (R-up-at-source / D38)", () = expect(msgs).toEqual([]); }); }); + +describe("C-8 intra-graph runtime rewire (R-rewire / D42)", () => { + it("surgical addDep (push-on-subscribe) → removeDep (drain) → idempotent setDeps; cache preserved", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 100 }); // B carries cached DATA + const aOnly = (ctx: Ctx) => ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + const sum = (ctx: Ctx) => + ctx.down([ + ["DATA", (ctx.depRecords[0].latest as number) + (ctx.depRecords[1].latest as number)], + ]); + const d = node([a], aOnly); + collect(d); + expect(d.cache).toBe(1); // (1) A settled → D ran + + d.addDep(b, sum); // (2) addDep(B): B cached → push-on-subscribe → D recomputes + expect(d.cache).toBe(101); // 1 + 100 + + b.down([["DATA", 50]]); // (3) B drives D + expect(d.cache).toBe(51); // 1 + 50 + + d.removeDep(a, aOnly); // (4) removeDep(A) → deps = [B] + expect(d.cache).toBe(51); // cache PRESERVED (A was not dirty — no recompute) + + a.down([["DATA", 9]]); // (5) A no longer drives D — its edge is drained + expect(d.cache).toBe(51); + + b.down([["DATA", 7]]); // B is dep0 now → drives D + expect(d.cache).toBe(7); + + let runs = 0; + const f = (ctx: Ctx) => { + runs++; + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }; + d.setDeps([b], f); // (6) setDeps to the current set → idempotent + expect(runs).toBe(0); // no spurious recompute + expect(d.cache).toBe(7); + }); + + it("rejects rewire on a terminal node (throw → graph-layer ERROR, D30)", () => { + const a = node([], null, { initial: 1 }); + const id = (ctx: Ctx) => ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + const d = node([a], id, { completeWhenDepsComplete: false }); + collect(d); + d.down([["COMPLETE"]]); // D terminal + expect(() => d.setDeps([a], id)).toThrow(/terminal/); + }); +}); diff --git a/packages/ts/src/__tests__/rewire.test.ts b/packages/ts/src/__tests__/rewire.test.ts new file mode 100644 index 00000000..286b6c82 --- /dev/null +++ b/packages/ts/src/__tests__/rewire.test.ts @@ -0,0 +1,192 @@ +/** + * Intra-graph runtime rewire — setDeps/addDep/removeDep (R-rewire / D42 / CSP-2.5). + * + * Focused substrate unit tests: each Q-semantic (push-on-subscribe, gate-preserve, + * cache-preserve, drain, reorder, idempotent, zero-deps) + each reject (self / cycle / + * terminal-this / non-resubscribable-terminal-dep / mid-fn). The exhaustive mid-wave + * interleavings are covered by the TLA+ model (~/src/graphrefly/formal/wave_rewire.tla). + */ + +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { type Node, node } from "../index.js"; + +function collect(n: Node) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const num = (ctx: Ctx, i: number): number => ctx.depRecords[i].latest as number; + +describe("rewire — Q-semantics (R-rewire / D42)", () => { + it("addDep wires a cached dep via push-on-subscribe and recomputes", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 100 }); + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + collect(d); + expect(d.cache).toBe(1); + + d.addDep(b, (ctx) => ctx.down([["DATA", num(ctx, 0) + num(ctx, 1)]])); + expect(d.cache).toBe(101); // 1 + 100 — b's cached DATA pushed on subscribe (R-push-subscribe) + }); + + it("addDep of a never-emitted (SENTINEL) dep does not re-arm the first-run gate (Q2)", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null); // no initial → SENTINEL, never emits + let runs = 0; + const d = node([a], (ctx) => { + runs++; + ctx.down([["DATA", num(ctx, 0) * 10]]); + }); + collect(d); + expect(d.cache).toBe(10); + runs = 0; + + d.addDep(b, (ctx) => { + runs++; + const bv = ctx.depRecords[1].latest; // SENTINEL = undefined — fn guards it + ctx.down([["DATA", num(ctx, 0) * 10 + (bv === undefined ? 0 : (bv as number))]]); + }); + expect(runs).toBe(0); // a SENTINEL added dep delivers START only — no recompute + + a.down([["DATA", 2]]); // gate NOT re-armed: a alone re-drives d without waiting for b + expect(runs).toBe(1); + expect(d.cache).toBe(20); // 2*10 + 0 (b still SENTINEL) + }); + + it("removeDep drains the removed dep + preserves cache (Q3/Q7)", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 2 }); + const sumAB = (ctx: Ctx) => ctx.down([["DATA", num(ctx, 0) + num(ctx, 1)]]); + const aOnly = (ctx: Ctx) => ctx.down([["DATA", num(ctx, 0)]]); + const d = node([a, b], sumAB); + collect(d); + expect(d.cache).toBe(3); + + d.removeDep(b, aOnly); + expect(d.cache).toBe(3); // cache preserved — removeDep of a non-dirty dep does not recompute + + b.down([["DATA", 99]]); // removed dep must NOT drive d (drained) + expect(d.cache).toBe(3); + + a.down([["DATA", 5]]); // d recomputes with the swapped (a-only) fn + expect(d.cache).toBe(5); + }); + + it("removeDep to zero deps → inert fn-no-deps, cache preserved (D42 SD-3)", () => { + const a = node([], null, { initial: 7 }); + let runs = 0; + const d = node([a], (ctx) => { + runs++; + ctx.down([["DATA", num(ctx, 0)]]); + }); + collect(d); + expect(d.cache).toBe(7); + runs = 0; + + d.removeDep(a, (ctx) => { + runs++; + ctx.down([["DATA", -1]]); + }); + expect(d.cache).toBe(7); // preserved + expect(runs).toBe(0); // inert — does not fire + + a.down([["DATA", 8]]); // a is no longer a dep + expect(d.cache).toBe(7); + expect(runs).toBe(0); + }); + + it("setDeps reorders kept deps without losing their state (Option-C, DepRecord-ref dispatch)", () => { + const a = node([], null, { initial: 10 }); + const b = node([], null, { initial: 20 }); + const f = (ctx: Ctx) => ctx.down([["DATA", num(ctx, 0) * 100 + num(ctx, 1)]]); + const d = node([a, b], f); + collect(d); + expect(d.cache).toBe(1020); // dep0=a=10, dep1=b=20 → 1020 + + d.setDeps([b, a], f); // reorder; kept-dep state (a=10, b=20) preserved + a.down([["DATA", 11]]); // a is now dep1; reroutes correctly + expect(d.cache).toBe(2011); // dep0=b=20, dep1=a=11 → 20*100+11 + }); + + it("setDeps to the current dep set is idempotent — no spurious recompute", () => { + const a = node([], null, { initial: 1 }); + let runs = 0; + const f = (ctx: Ctx) => { + runs++; + ctx.down([["DATA", num(ctx, 0)]]); + }; + const d = node([a], f); + collect(d); + expect(d.cache).toBe(1); + runs = 0; + + d.setDeps([a], f); + expect(runs).toBe(0); + expect(d.cache).toBe(1); + }); + + it("addDep returns the new dep index", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 2 }); + const f = (ctx: Ctx) => ctx.down([["DATA", num(ctx, 0)]]); + const d = node([a], f); + collect(d); + expect(d.addDep(b, f)).toBe(1); + expect(d.addDep(b, f)).toBe(1); // already present → still index 1 (fn swap only) + }); +}); + +describe("rewire — rejects (R-rewire / D42)", () => { + it("rejects a self-dependency", () => { + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + collect(d); + expect(() => d.setDeps([d as Node], (ctx) => ctx.down([["DATA", 0]]))).toThrow( + /self-dependency/, + ); + }); + + it("rejects a cycle", () => { + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + const e = node([d], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + collect(e); // e depends on d (→ a) + // adding e as a dep of d would close d→e→d + expect(() => d.addDep(e, (ctx) => ctx.down([["DATA", num(ctx, 0)]]))).toThrow(/cycle/); + }); + + it("rejects rewire on a terminal node", () => { + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]]), { + completeWhenDepsComplete: false, + }); + collect(d); + d.down([["COMPLETE"]]); // d terminal + expect(() => d.setDeps([a], (ctx) => ctx.down([["DATA", 0]]))).toThrow(/terminal/); + }); + + it("rejects adding a non-resubscribable terminal dep", () => { + const term = node([], (ctx) => ctx.down([["COMPLETE"]])); // producer → terminal on activation + collect(term); + expect(term.status).toBe("completed"); + + const a = node([], null, { initial: 1 }); + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + collect(d); + expect(() => d.addDep(term, (ctx) => ctx.down([["DATA", num(ctx, 0)]]))).toThrow( + /terminal dep/, + ); + }); + + it("rejects mid-fn rewire (a fn mutating its own deps mid-wave = D37 feedback cycle)", () => { + const a = node([], null, { initial: 1 }); + const x = node([], null, { initial: 9 }); + let dh: Node | undefined; + dh = node([a], (ctx) => { + dh?.addDep(x, (c) => c.down([["DATA", num(c, 0)]])); + ctx.down([["DATA", num(ctx, 0)]]); + }); + expect(() => collect(dh as Node)).toThrow(/mid-fn|feedback/); + }); +}); diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index b1aee710..cd373ec3 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -66,8 +66,9 @@ export interface NodeOptions { const defaultEquals = Object.is as (a: unknown, b: unknown) => boolean; export class Node { - private readonly _deps: Node[]; - private readonly _handle: Handle | null; + private _deps: Node[]; + private _handle: Handle | null; + private readonly _pool: "sync" | "async"; private readonly _dispatcher: Dispatcher; private readonly _equals: (a: T, b: T) => boolean; private readonly _partial: boolean; @@ -84,6 +85,9 @@ export class Node { private _subscribers = new Set(); private _activated = false; private _depUnsubs: Array<() => void> = []; + // R-rewire: each dep's subscription reads its index from a mutable box so a surgical + // rewire-reorder reroutes in O(1) (no per-message indexOf scan — F-PERF for high fan-in). + private _depIdxBoxes: Array<{ v: number }> = []; // per-dep wave state private _depBatch: Array; @@ -100,6 +104,8 @@ export class Node { private _hasCalledFnOnce = false; private _emittedDirtyThisWave = false; private _insideRunWave = false; + /** R-rewire: reentrancy guard for setDeps/addDep/removeDep (one mutation in flight). */ + private _inDepMutation = false; /** Node's own terminal: undefined = live, true = COMPLETE, else ERROR payload. */ private _terminal: true | unknown | undefined = undefined; private _hasTorndown = false; @@ -142,11 +148,12 @@ export class Node { this._pausable = opts.pausable ?? true; this._replayN = opts.replayBuffer ?? 0; this._dynamic = opts.dynamic ?? false; + this._pool = opts.pool ?? "sync"; this.name = opts.name; if (handleOrFn === null) this._handle = null; else if (typeof handleOrFn === "function") - this._handle = this._dispatcher.register(handleOrFn, opts.pool ?? "sync"); + this._handle = this._dispatcher.register(handleOrFn, this._pool); else this._handle = handleOrFn; const n = deps.length; @@ -231,25 +238,220 @@ export class Node { this._up(msgs); } + // ── rewire (R-rewire / D42): intra-graph runtime topology mutation ── + + /** + * Replace this node's deps atomically (surgical, Option-C). Requires an explicit + * `fn` (SD-1 fn-deps pairing — user fns read depRecords positionally). Kept deps + * keep their subscription + per-dep state; only removed deps unsubscribe and only + * added deps fresh-subscribe (push-on-subscribe for an added cached dep). The + * first-run gate and cache are PRESERVED (R-rewire Q2/Q7). Intra-graph only (D22). + */ + setDeps(newDeps: Node[], fn: NodeFn): void { + this._rewire(this._dedupDeps(newDeps), fn); + } + + /** Add one dep (special case of setDeps); returns its index. fn required (SD-1). */ + addDep(depNode: Node, fn: NodeFn): number { + const next = this._deps.includes(depNode) ? [...this._deps] : [...this._deps, depNode]; + this._rewire(next, fn); + return this._deps.indexOf(depNode); + } + + /** Remove one dep (special case of setDeps); idempotent if absent (fn swap still applies). */ + removeDep(depNode: Node, fn: NodeFn): void { + this._rewire( + this._deps.filter((d) => d !== depNode), + fn, + ); + } + + private _dedupDeps(deps: Node[]): Node[] { + const seen = new Set>(); + const out: Node[] = []; + for (const d of deps) + if (!seen.has(d)) { + seen.add(d); + out.push(d); + } + return out; + } + + /** Is `target` reachable upstream from `from` (following deps)? Cycle-prevention DFS. */ + private _reachableUpstream(from: Node, target: Node): boolean { + const seen = new Set>(); + const stack: Node[] = [from]; + while (stack.length > 0) { + const n = stack.pop(); + if (n === undefined) continue; + if (n === target) return true; + if (seen.has(n)) continue; + seen.add(n); + for (const d of n._deps) stack.push(d); + } + return false; + } + + private _rewire(newDeps: Node[], fn: NodeFn): void { + // ── rejects (R-rewire / D42) ── + if (this._terminal !== undefined) + throw new Error( + "rewire: node is terminal (completed/errored) — cannot rewire (R-rewire / D42)", + ); + if (this._insideRunWave) + throw new Error( + "rewire: mid-fn topology mutation — a fn mutating its own deps mid-wave is the feedback cycle (R-rewire / D37)", + ); + if (this._inDepMutation) + throw new Error( + "rewire: reentrant dep mutation — another setDeps/addDep/removeDep is in flight (R-rewire)", + ); + if (newDeps.includes(this as unknown as Node)) + throw new Error("rewire: self-dependency rejected (R-rewire / D42)"); + const oldDeps = this._deps; + const added = newDeps.filter((d) => !oldDeps.includes(d)); + for (const d of added) { + if (this._reachableUpstream(d, this as unknown as Node)) + throw new Error( + "rewire: would create a cycle — dep already transitively depends on this node (R-rewire / D42)", + ); + if (d._terminal !== undefined && !d._resubscribable) + throw new Error( + "rewire: cannot add a non-resubscribable terminal dep — would wedge (R-rewire / D42)", + ); + } + + this._inDepMutation = true; + try { + // fn swap (SD-1): re-register against the same pool. + this._handle = this._dispatcher.register(fn, this._pool); + + const removed = oldDeps.filter((d) => !newDeps.includes(d)); + let removedDirtyContributor = false; + for (const d of removed) { + const oldIdx = oldDeps.indexOf(d); + if (this._depDirty[oldIdx]) { + removedDirtyContributor = true; + this._pending--; + } + if (this._activated) { + const box = this._depIdxBoxes[oldIdx]; + if (box) box.v = -1; // drain: any stale in-flight callback drops + const unsub = this._depUnsubs[oldIdx]; + if (unsub) unsub(); // stops the removed dep's edge — no further delivery + } + } + + // Rebuild per-dep parallel arrays in newDeps order; kept deps carry their state + // + subscription, added deps start fresh (R-rewire Q1/Q4). + const n = newDeps.length; + const newBatch: Array = new Array(n).fill(null); + const newPrev: unknown[] = new Array(n).fill(SENTINEL); + const newHasData: boolean[] = new Array(n).fill(false); + const newDirty: boolean[] = new Array(n).fill(false); + const newTier: number[] = new Array(n).fill(0); + const newTerminal: Array = new Array(n).fill(undefined); + const newUnsubs: Array<() => void> = new Array(n); + const newBoxes: Array<{ v: number }> = new Array(n); + for (let j = 0; j < n; j++) { + const oldIdx = oldDeps.indexOf(newDeps[j]); + if (oldIdx !== -1) { + newBatch[j] = this._depBatch[oldIdx]; + newPrev[j] = this._depPrev[oldIdx]; + newHasData[j] = this._depHasData[oldIdx]; + newDirty[j] = this._depDirty[oldIdx]; + newTier[j] = this._depTier[oldIdx]; + newTerminal[j] = this._depTerminal[oldIdx]; + newUnsubs[j] = this._depUnsubs[oldIdx]; + // carry the kept dep's subscription box and point it at the new index (O(1) reroute) + const box = this._depIdxBoxes[oldIdx]; + if (box) box.v = j; + newBoxes[j] = box; + } + } + this._deps = newDeps; + this._depBatch = newBatch; + this._depPrev = newPrev; + this._depHasData = newHasData; + this._depDirty = newDirty; + this._depTier = newTier; + this._depTerminal = newTerminal; + this._depUnsubs = newUnsubs; + this._depIdxBoxes = newBoxes; + // depRecords are wave-scratch (rebuilt in _buildCtx); fresh array + drop the + // cached sync ctx whose `depRecords` pointed at the old array. + this._depRecords = newDeps.map(() => ({ + batch: null, + prevData: SENTINEL, + latest: SENTINEL, + tier: 0, + terminal: undefined, + })); + this._syncCtx = null; + + // Subscribe added deps — push-on-subscribe (R-push-subscribe) delivers a cached + // dep's DATA here, which drives _maybeRun; a SENTINEL dep delivers START only. + if (this._activated) { + for (const d of added) this._subscribeDepAt(d); + } + + // Q6 auto-settle: removing the sole dirty contributor closes the wave. With deps + // remaining, recompute via the normal gate (equals absorbs a no-change recompute → + // RESOLVED); with zero deps the node is inert (degenerate fn-no-deps) so just + // un-dirty downstream with a RESOLVED. Cache is preserved either way (Q7). + if (removedDirtyContributor && this._pending === 0 && this._status === "dirty") { + if (newDeps.length > 0) { + this._maybeRun(); + } else { + this._status = this._hasData ? "settled" : "sentinel"; + if (this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = false; + this._emitToSubs(["RESOLVED"]); + } + } + } + } finally { + this._inDepMutation = false; + } + } + // ── activation / deactivation (lazy; R-rom-ram) ── private _activate(): void { this._activated = true; - for (let i = 0; i < this._deps.length; i++) { - const idx = i; - const unsub = this._deps[i].subscribe((msg) => this._receiveFromDep(idx, msg)); - this._depUnsubs.push(unsub); - } + this._depUnsubs = new Array(this._deps.length); + this._depIdxBoxes = new Array(this._deps.length); + for (const dep of this._deps) this._subscribeDepAt(dep); // Depless producer (fn, no deps): run once on activation. if (this._deps.length === 0 && this._handle !== null && !this._hasCalledFnOnce) { this._runWave(); } } + /** + * Subscribe to a dep. The dispatch callback reads the dep's CURRENT index from a + * mutable box (O(1)); a surgical rewire that reorders kept deps just updates the box + * (R-rewire Option-C / D42) — no re-subscribe, no per-message indexOf scan. A removed + * dep's box is set to -1 so any stale in-flight callback drops (drain). + */ + private _subscribeDepAt(depNode: Node): void { + const idx0 = this._deps.indexOf(depNode); + const box = { v: idx0 }; + const unsub = depNode.subscribe((msg) => { + if (box.v === -1) return; // dep removed — stale callback, drop (drain) + this._receiveFromDep(box.v, msg); + }); + if (idx0 !== -1) { + this._depUnsubs[idx0] = unsub; + this._depIdxBoxes[idx0] = box; + } + } + private _deactivate(): void { this._activated = false; - for (const u of this._depUnsubs) u(); + for (const u of this._depUnsubs) if (u) u(); this._depUnsubs = []; + this._depIdxBoxes = []; for (const fn of this._onDeactivation) fn(); this._onDeactivation = []; this._onInvalidate = []; @@ -761,8 +963,9 @@ export class Node { /** R-terminal: resubscribable reset clears terminal + dep state + re-arms the gate. */ private _resetLifecycle(): void { - for (const u of this._depUnsubs) u(); + for (const u of this._depUnsubs) if (u) u(); this._depUnsubs = []; + this._depIdxBoxes = []; this._activated = false; this._terminal = undefined; this._hasTorndown = false; From 44e9d54fb9ad118d2d251b093468c7df1b4da422 Mon Sep 17 00:00:00 2001 From: David Chen Date: Fri, 29 May 2026 08:08:54 -0700 Subject: [PATCH 007/175] feat(qa): enhance QA workflow documentation and improve rewire functionality - Updated the QA workflow documentation to reflect the clean-slate redesign of GraphReFly, clarifying repo detection and operational context. - Introduced new tests for rewire functionality, ensuring atomic settles and proper DIRTY/DATA emissions during dependency changes. - Enhanced the Node class to support atomic two-phase settles after dependency mutations, improving performance and correctness in rewire scenarios. - Added comprehensive test cases to validate the behavior of the rewire mechanism, including edge cases for dependency management. --- .claude/skills/qa/SKILL.md | 206 ++++++----------------- packages/ts/src/__tests__/rewire.test.ts | 65 +++++++ packages/ts/src/node/node.ts | 62 +++++-- 3 files changed, 171 insertions(+), 162 deletions(-) diff --git a/.claude/skills/qa/SKILL.md b/.claude/skills/qa/SKILL.md index a177d91a..b5c40454 100644 --- a/.claude/skills/qa/SKILL.md +++ b/.claude/skills/qa/SKILL.md @@ -4,201 +4,107 @@ description: "Adversarial code review, apply fixes, final checks (test/lint/buil argument-hint: "[--skip-docs] [optional context about what was implemented]" --- -You are executing the **qa** workflow for **GraphReFly** (cross-language: TypeScript + Python + Rust port). +You are executing the **qa** workflow for the **clean-slate GraphReFly** redesign. -Operational docs live in **graphrefly-ts** (this repo). The diff may include changes in `graphrefly-ts`, `graphrefly-py` (`~/src/graphrefly-py`), or **`graphrefly-rs`** (`~/src/graphrefly-rs` — the Rust port). +This repo is **`@graphrefly/ts`** — the self-contained TypeScript implementation (D32); clean-slate code lives in **`packages/ts/src/`**. Siblings (each self-contained, cross-language = wire bridge, never in-process): `@graphrefly/rust` (`~/src/graphrefly-rs`), `@graphrefly/py` (`~/src/graphrefly-py`). The language-neutral authority (spec / decisions / plan / conformance / formal) lives in **`~/src/graphrefly`** (branch `clean-slate`) as jsonl — when this skill and that repo disagree, that repo wins (CLAUDE.md). -### Repo detection - -Inspect the diff to detect which repo(s) are touched. If the diff includes paths under `~/src/graphrefly-rs/` (or the working directory IS `~/src/graphrefly-rs`), this is a **Rust-port QA pass** — see "Rust-port QA additions" callouts inline below for the doc reads, subagent prompt extensions, final-check commands, and Phase 4 doc updates that apply. +> **Stale-infra guard.** Do NOT reach for the retired port-model surfaces: `packages/pure-ts/**` (frozen read-only reference only, D41), `docs/implementation-plan.md` / `implementation-plan-13.6-*.md` / `optimizations.md` / `roadmap.md` / `test-guidance.md` / `docs-guidance.md`, `GRAPHREFLY-SPEC.md` / `COMPOSITION-GUIDE.md` (migrated to `spec/rules.jsonl` + `guide/guide.jsonl`, B7), the `@graphrefly/graphrefly` shim + its 4-file subpath rule, the `Impl`/facade/actor model, the Rust `migration-status.md` / `porting-deferred.md` / canonical-spec-13.6 registries. The clean-slate authority is the `~/src/graphrefly` jsonl. Context from user: $ARGUMENTS ### Flag detection - If `$ARGUMENTS` contains `--skip-docs`, skip Phase 4 (Documentation Updates). +### Repo detection +Inspect the diff to detect which package(s) are touched: `packages/ts/` (this repo), `~/src/graphrefly-py`, `~/src/graphrefly-rs`. The cross-language contract is **behavioral conformance (D24)**, not symbol parity — review each arm against the SAME `spec/rules.jsonl` + `spec/conformance.jsonl`, per-language idioms aside. + --- ## Phase 1: Adversarial Code Review ### 1a. Gather the diff -Run `git diff` to get all uncommitted changes. If there are also untracked files relevant to the task, read and include them. - -**Rust-port QA additions** — if the diff is in `~/src/graphrefly-rs/`, ALSO read these as part of the context-load (they are the canonical authority + active deferred-concerns registry for the Rust port; QA must NOT contradict them, and SHOULD update them when the slice closes a deferred item): +Run `git diff` for uncommitted changes; if the chat's work was already committed, diff against the chat's baseline commit (`git log --oneline` to find it, then `git diff ..HEAD`). Include relevant untracked files (read them). Concentrate the review on the **substantive hand-written code** — formally-verified TLA+ (already TLC-checked), generated artifacts, and jsonl data are lower bug-risk than imperative substrate/graph code. -- **`docs/implementation-plan-13.6-canonical-spec.md`** (graphrefly-ts) — single-document canonical spec post-Phase 13.6.A. The Rust port's behavior authority. Use rule IDs (`R`) to cite spec rules in findings. -- **`docs/implementation-plan-13.6-flowcharts.md`** (graphrefly-ts) — Mermaid diagrams for all internal methods/processes. Cross-reference for call/data-flow shape during edge-case review. -- **`~/src/graphrefly-rs/docs/migration-status.md`** — milestone tracker. Read to know which slice this QA pass covers, what's claimed-as-landed, and what test counts to verify. -- **`~/src/graphrefly-rs/docs/porting-deferred.md`** — registry of audit-surfaced concerns deferred to evidence-driven slices. **DO NOT raise findings that match an existing deferred entry** — those are already-acknowledged divergences/limitations. DO raise findings that contradict a deferred entry's stated scope (e.g., a "deferred for now" item that the slice actually starts touching but didn't update). +Also load the clean-slate context the review must NOT contradict: +- `~/src/graphrefly/spec/rules.jsonl` — the protocol 宪法 (R-* rules); the behavior authority. Cite R-ids in findings. +- `~/src/graphrefly/decisions/decisions.jsonl` — the governing D# (or `/decision-guard`); the F-* floor + durable values. +- `~/src/graphrefly/plan/backlog.jsonl` + `plan/antipatterns.jsonl` — **already-acknowledged deferred concerns (B#) and known anti-patterns. DO NOT raise a finding that matches an existing deferred B# or antipattern** — those are accepted. DO raise a finding that *contradicts* a deferred entry's stated scope. +- `~/src/graphrefly/spec/conformance.jsonl` — the C-* scenarios the change must keep green (+ their `runtimes` status). +- `~/src/graphrefly/formal/*.tla` — when the change implements a formally-modeled rule, cross-check the impl against the TLC-verified model. ### 1b. Launch parallel review subagents -Launch these as parallel Agent calls. Each receives the diff and the context from $ARGUMENTS (what was implemented and why). +Launch as parallel Agent calls. Each receives the diff + the context from $ARGUMENTS (what was implemented and why). Tell each to do a STATIC review (no servers, no test runs) and return ONLY a findings list. -**Subagent 1: Blind Hunter** — Pure code review, no project context: -> You are a Blind Hunter code reviewer. Review this diff for: logic errors, off-by-one errors, race conditions, resource leaks, missing error handling, security issues, dead code, unreachable branches. For Python code, also check thread safety (including free-threaded Python without GIL). Output each finding as: **title** | **severity** (critical/major/minor) | **location** (file:line) | **detail**. Be adversarial — assume bugs exist. +**Subagent 1: Blind Hunter** — pure code review, no project context: +> You are a Blind Hunter code reviewer. Review this diff for: logic errors, off-by-one, race/re-entrancy hazards, resource leaks (unclosed subscriptions, unbounded registries), stale-closure / index-desync bugs, missing error handling, dead/unreachable code, sparse-array holes, security issues. For Python, also check thread/free-threaded safety. Be adversarial — assume bugs exist; trace the suspicious paths concretely. If a suspicious path is actually correct, say so in one line rather than raising noise. Output each finding as: **title** | **severity** (critical/major/minor) | **location** (file:line) | **detail** (trigger + consequence + suggested fix). -**Subagent 2: Edge Case Hunter** — Has project read access: -> You are an Edge Case Hunter. Review this diff in the context of **GraphReFly** (`~/src/graphrefly/GRAPHREFLY-SPEC.md`). First, read `archive/optimizations/cross-language-notes.jsonl` and collect all entries with `id` prefix `divergence-` — these are **confirmed intentional cross-language divergences** that must NOT be raised as findings. +**Subagent 2: Edge Case Hunter** — clean-slate spec-aware: +> You are an Edge Case Hunter reviewing a change against the GraphReFly clean-slate SPEC. The authority is `~/src/graphrefly` jsonl (branch clean-slate) — NOT any docs/*.md or packages/pure-ts (retired port-model; ignore). Read the relevant `spec/rules.jsonl` R-* rules + `decisions/decisions.jsonl` D# for the area under review, and the matching `spec/conformance.jsonl` C-* + `formal/*.tla` model if the change implements a spec-locked behavior. > -> **Post-Phase-13.9.A repo layout (read once before reviewing TS diffs):** the pure-TS implementation lives at `packages/pure-ts/src/` (NOT `src/`). Root `src/` is the `@graphrefly/graphrefly` shim — every file there is a one-liner `export * from "@graphrefly/pure-ts/"`. If a TS diff touches root `src/`, that's almost always a public-API surface widening (a new subpath was added) and the review should verify 4-file consistency: (1) the actual source under `packages/pure-ts/src//`, (2) `packages/pure-ts/tsup.config.ts` `ENTRY_POINTS` (+ `nodeOnlyEntries` if Node-only), (3) `packages/pure-ts/package.json` `exports`, (4) the matching one-liner shim source at root `src//` plus its registration in root `tsup.config.ts` `ENTRY_POINTS` and root `package.json` `exports`. If the diff also affects cross-impl behavior, the review should also confirm a corresponding parity scenario landed in `packages/parity-tests/scenarios//`. +> Check protocol/wave invariants against the rules: message tuples `[[Type,Data?]]`, one array = one wave (R-msg-format); DIRTY-before-DATA in the same wave (R-dirty-before-data); two-phase glitch-free diamond, recompute-once (R-two-phase/R-diamond); ctx.up control-tier-only (R-ctx-up); SENTINEL = absence-of-DATA, never-emitted detector `prevData===undefined` (R-sentinel); equals DATA→RESOLVED only on a single-DATA wave (R-equals); first-run gate (R-first-run-gate); INVALIDATE idempotent + lifecycle-continue (R-invalidate-idempotent); terminal-is-forever / resubscribable reset (R-terminal); ROM/RAM cache (R-rom-ram); PAUSE lockset + modes (R-pause-lockset/R-pause-modes); reentrancy reject (R-reentrancy/D37). > -> Then check: unhandled message sequences (DIRTY without follow-up, DATA vs RESOLVED), diamond resolution (recompute once), COMPLETE/ERROR terminal rules, forward-unknown-types, batch semantics (DATA deferred, DIRTY not), reconnect/teardown leaks, meta companion nodes, and graph mount/signal propagation when `Graph` is in scope. Also flag violations of design invariants (spec §5.8–5.12): polling patterns (busy-wait or setInterval/time.sleep loops on node values), imperative triggers bypassing graph topology, bare Promises/queueMicrotask/setTimeout (TS) or asyncio.ensure_future/create_task/threading.Timer (PY) for reactive scheduling, direct Date.now()/performance.now() (TS) or time.time_ns()/time.monotonic_ns() (PY) usage (must use `core/clock.ts` or `core/clock.py` — for TS, that's now `packages/pure-ts/src/core/clock.ts`), hardcoded message type checks instead of messageTier/message_tier utilities, and Phase 4+ APIs that leak protocol internals (DIRTY/RESOLVED/bitmask) into their primary surface. **If the change touches `packages/pure-ts/src/patterns/` or `packages/pure-ts/src/compat/`, verify the implementation against COMPOSITION-GUIDE.md categories (§1 lazy activation, §2 subscription ordering, §3 null guards, §5 wiring order, §7 feedback cycles, §8 SENTINEL gate).** **Browser / Node / Universal tier (TS):** if the change adds or moves code in `packages/pure-ts/src/extra/` or `packages/pure-ts/src/patterns/`, confirm (a) any new `node:*` import or `require("")` / `fileStorage` / `sqliteStorage` / `child_process` / filesystem API lives in a `/node` subpath source file, not on a universal path; (b) any new DOM global (`window`, `document`, `indexedDB`, `Worker`, `MessagePort` constructor calls) lives in a `/browser` subpath; (c) new subpaths are registered in both `packages/pure-ts/tsup.config.ts` `ENTRY_POINTS` (+ `nodeOnlyEntries` when node-only) and `packages/pure-ts/package.json` `exports`, AND mirrored in the root shim's `tsup.config.ts` + `package.json` `exports` + a one-liner shim source at root `src//.ts`; (d) JSDoc `@example` blocks import from the correct subpath via the public `@graphrefly/graphrefly/...` name (NOT `@graphrefly/pure-ts` directly) — a Node-only adapter must not suggest the universal barrel in its example. The `assertBrowserSafeBundles` guardrail lives in `packages/pure-ts/tsup.config.ts` (NOT in the root shim's tsup, which has no source-level imports to police). See `docs/docs-guidance.md` § "Browser / Node / Universal split" for the convention. **If the diff is in `~/src/graphrefly-rs/` (Rust port):** review against the *single-document canonical spec* at `~/src/graphrefly-ts/docs/implementation-plan-13.6-canonical-spec.md` (NOT the older multi-file TS spec — they diverge per §11 Implementation Deltas). Cross-reference rule IDs (`R`) in findings. Cross-reference call/data-flow shape via `~/src/graphrefly-ts/docs/implementation-plan-13.6-flowcharts.md`. Read `~/src/graphrefly-rs/docs/porting-deferred.md` and DROP findings that match an already-acknowledged deferred entry (perf-tier §10 simplifications, v1 dispatcher limitations, spec divergences acknowledged in v1, Phase 13.8 carry-forwards). For Rust slices that close milestones in the `packages/parity-tests/README.md` schedule (M1 dispatcher, M2 Slice E Graph, M3 operators, M4 storage, M5 structures), also flag whether `packages/parity-tests/scenarios//` was widened with corresponding `describe.each(impls)` scenarios — surface coverage of the new milestone is part of the parity-gate definition. Also flag Rust-specific invariants: `unsafe` usage (forbidden — `#![forbid(unsafe_code)]`); `unwrap()` / `expect()` on user-facing paths (only allowed in tests/build scripts/impossible-by-construction with comment); missing `#[must_use]` on public-fn returns; raw integer types where newtypes (`NodeId(u64)`, `HandleId(u64)`, `FnId(u64)`, `LockId(u64)`) should be used; refcount imbalance via `BindingBoundary::retain_handle` / `release_handle` pairs; lock-discipline asymmetry across the `parking_lot::Mutex` boundary (sink-fire-with-lock-held vs lock-released); `Send + Sync` violations on public types; async runtime introduction in `graphrefly-core` (forbidden — Core stays sync). For each finding: **title** | **trigger_condition** | **potential_consequence** | **location** | **suggested_guard**. - -### 1c. Triage findings - -Classify each finding into: -- **patch** — fixable code issue. Include the fix recommendation. -- **defer** — pre-existing issue, not caused by this change. -- **reject** — false positive or noise. Drop silently. +> Flag floor violations: imperative side-channel triggers (R-no-imperative — emitters/callbacks/timers+set instead of ctx.up/message flow); polling/busy-wait (R-no-polling); bare async in the sync core (R-no-raw-async / F-SYNC-CORE — async only in sources / pool / wire bridge); inline-fn bypassing the dispatcher (R-dispatch-all / F-DISPATCH-ALL); peeking a dep `.cache` to seed compute (R-data-not-peek); hardcoded `type === "DATA"` instead of messageTier (R-tier); protocol internals (DIRTY/RESOLVED/bitmask) leaking into value-level sugar (R-primary-api-clean / DR-1); counters/inspection on the thin node (R-node-thin); a new verb (D4 closed set) or a 10th tier (D9) introduced casually; graph-level shared mutable state accessed implicitly instead of an explicit node+dep (D22/D23); cross-graph in-process coupling instead of a wire bridge (D22/D32). +> +> If the change implements a formally-modeled rule, identify any place the impl DIVERGES from the `formal/*.tla` model (cite the invariant). Surface real-but-unmodeled cross-axis interactions (e.g. X×batch, X×pause) and say whether each is a genuine gap or acceptably-deferred. DROP any finding that matches an already-acknowledged `plan/backlog.jsonl` B# or `plan/antipatterns.jsonl` entry. Output each finding as: **title** | **severity** | **location** (file:line or R-id) | **detail** (the rule/D# it relates to + what the impl does + divergence/gap/ok). -For each **patch** and **defer** finding, evaluate fix priority (most to least important): -1. **Spec alignment** — matches `~/src/graphrefly/GRAPHREFLY-SPEC.md` (or `docs/implementation-plan-13.6-canonical-spec.md` for Rust-port diffs — the canonical post-Phase 13.6.A consolidated spec) -2. **Semantic correctness** — protocol and node contract -3. **Completeness** — edge cases covered -4. **Consistency** — patterns elsewhere in graphrefly-ts (or graphrefly-rs for Rust-port diffs) -5. **Level of effort** +Scale the reviewer count to the change: 2 is the default; for a large or high-risk substrate change add a third reviewer on a specific axis (e.g. concurrency/pool, or a perspective-diverse second spec reviewer). -**Optional:** Compare tricky operator behavior with **callbag-recharge** at `~/src/callbag-recharge` for precedent — GraphReFly still wins on spec conflicts. +### 1c. Triage findings +Classify each: **patch** (fixable, caused by this change — include the fix) · **defer** (pre-existing or out-of-scope — note it) · **reject** (false positive / noise — drop silently). Cross-check every finding against `plan/backlog.jsonl` + `plan/antipatterns.jsonl`; a match to an accepted deferral → **reject** silently. -**Rust-port QA additions** — when triaging findings on `~/src/graphrefly-rs/` diffs: -- Cross-check every finding against `~/src/graphrefly-rs/docs/porting-deferred.md`. Findings that match an already-acknowledged deferred entry (perf-tier §10, v1 dispatcher limitation, spec divergence acknowledged in v1, Phase 13.8 carry-forward) → **reject** silently. -- Findings that contradict the canonical spec at `~/src/graphrefly-ts/docs/implementation-plan-13.6-canonical-spec.md` → **patch** (high priority — canonical wins over current TS impl per §11 Implementation Deltas). -- Findings about TS-vs-Rust behavior gaps → check whether TS is the reference or canonical is. The canonical spec wins; if Rust is closer to canonical than TS, the finding is `reject`. -- Findings about Rust-specific invariants (`unsafe` usage, refcount imbalance, `Send + Sync`, `unwrap` on user-facing paths) → **patch** (Rust's value over TS / PY is compiler-enforced safety; bypassing forfeits the win). +Fix priority (most→least): 1) **spec alignment** (`spec/rules.jsonl` / the F-* floor — a rule wins over current impl) · 2) **semantic correctness** (protocol + node contract) · 3) **completeness** (edge cases) · 4) **consistency** (patterns already in `packages/ts/src/`) · 5) **level of effort**. (Frozen `packages/pure-ts/**` + `~/src/callbag-recharge` are read-only precedent only — the clean-slate spec wins on any conflict.) ### 1d. Present findings (HALT) +Present ALL patch + defer findings (treat equally). For each: the issue + location, the **recommended fix** (pros/cons), whether it affects architecture, and whether it needs a user decision or can be auto-applied. Group: +1. **Needs Decision** — architecture-affecting or ambiguous (route per the floor: architectural lock → `/design-review` → D#; wave-protocol behavior change → `/spec-amend`; an open question with no clear answer → `plan/backlog.jsonl` B#). +2. **Auto-applicable** — clear fixes following existing patterns. -Present ALL patch and defer findings to the user. Treat both equally. For each finding: -- The issue and its location -- **Recommended fix** with pros/cons -- Whether it affects architecture (flag these) -- Whether it needs user decision or can be auto-applied - -Group findings: -1. **Needs Decision** — architecture-affecting or ambiguous fixes -2. **Auto-applicable** — clear fixes that follow existing patterns - -**Cross-language decision log:** For **Needs Decision** items that are architectural or affect TS/Python parity, add them to **`docs/optimizations.md`** under "Active work items" (this repo is the single source of truth for both TS and PY). When resolved, archive to `archive/optimizations/resolved-decisions.jsonl` per `docs/docs-guidance.md` § "Optimization decision log". - -**Wait for user decisions on group 1. Group 2 can be applied immediately if user approves the batch.** +**Wait for user decisions on group 1.** Group 2 may be applied on the user's batch approval. Do NOT silently pick on a needs-decision item (no-autonomous-decisions). --- ## Phase 2: Apply Review Fixes - -Apply the approved fixes from Phase 1. +Apply the approved fixes. Cite the governing R-id / D# in any new test expectations. --- ## Phase 3: Final Checks +Run all checks for the affected package(s) and fix failures (do NOT skip/ignore): -Run all checks for the affected repo(s) and fix any failures (do NOT skip or ignore): - -**TypeScript (post-Phase-13.9.A cleave):** -1. `pnpm test` — runs `pnpm --filter @graphrefly/pure-ts test && pnpm --filter @graphrefly/parity-tests test`. All tests must pass. Use `pnpm test:pure-ts` or `pnpm test:parity` to scope; `pnpm --filter @graphrefly/pure-ts test:watch` for watch mode. -2. `pnpm run lint:fix` — fix lint issues (workspace-wide; biome is shared at root). -3. `pnpm run build` — runs `pnpm --filter @graphrefly/pure-ts build && tsup` (pure-ts first, then root shim). The pure-ts build's `onSuccess` runs `assertBrowserSafeBundles` (fails the build with a `via X → Y → Z` chain if any universal entry transitively imports `node:*` or a bare Node builtin). The shim's tsup config has no equivalent guardrail (intentional — the shim has no source-level node-or-DOM imports, just `export *` lines). If `assertBrowserSafeBundles` fails, move the offending symbol to a `/node` subpath per `docs/docs-guidance.md` § "Browser / Node / Universal split", don't silence the guardrail. **Adding a new public subpath is a 4-file change post-cleave**: source under `packages/pure-ts/src//`, pure-ts `tsup.config.ts` `ENTRY_POINTS` + `package.json` `exports`, and matching shim entries (root `tsup.config.ts` `ENTRY_POINTS` + root `package.json` `exports` + a one-liner shim source at root `src/.ts`). - -**Python (if PY code was changed):** -1. `cd ~/src/graphrefly-py && uv run pytest` -2. `cd ~/src/graphrefly-py && uv run ruff check --fix src/ tests/` -3. `cd ~/src/graphrefly-py && uv run ruff format src/ tests/` -4. `cd ~/src/graphrefly-py && uv run mypy src/` - -**Rust (if Rust code was changed in `~/src/graphrefly-rs/`):** - -1. `cd ~/src/graphrefly-rs && mise run gate` — **the sanctioned QA gate.** ONE - command runs fmt --check → clippy (default-members) → `cargo nextest run - --profile ci` (incl. the `cascade_depth` stack-safety guards) *sequentially* - under an atomic mutex with process-group teardown, a direct log, a - self-timeout diagnostic, and a GUARANTEED terminal sentinel. Do **NOT** - hand-run the 4 cargo commands separately in a QA pass — concurrent/stacked - cargo invocations fighting the single target lock is the exact swap-thrash - foot-gun this gate exists to prevent (`feedback_no_chained_background_cargo.md`). - Use `mise run gate:core` for the fast `-p graphrefly-core`-scoped variant - during iteration; the **full `mise run gate` is required before the slice - closes**. `GATE_CLIPPY_DENY=1 mise run gate` to treat clippy warnings as - errors. (The gate runs `cargo-nextest`, never legacy `cargo test`.) -2. **Monitor it by the sentinel, never a guessed string.** `mise run gate` - emits `<<>> exit=… reason=ok|fail|timeout|signal|crash …` - on *every* terminal path (stdout AND the `.gate/*.log`). A Monitor's - until-condition MUST grep that token — never harness-buffered tool output, - never a non-guaranteed progress line. **Subagent hygiene:** if this QA pass - runs in a spawned subagent, run the gate **synchronously** (foreground, wait - for the sentinel) or tear down any backgrounded command (kill by process - group) **before returning** — a live background process leaks as a stale - parent-session "running" entry indistinguishable from a real hang. Full - rationale + the "is it hung?" checklist + the disk/`target/`-growth - tradeoff: `~/src/graphrefly-ts/docs/test-guidance.md` § "Running long - commands reliably / diagnosing a stuck run" (memories - `feedback_long_command_observation.md`, `feedback_subagent_bg_hygiene.md`). -3. **Not covered by the gate — check these manually:** - - Loom: `cargo test -p graphrefly-core --features loom-checked` (needs the - `--cfg loom` build; nextest can't run it). Run via - `mise run run-logged -- cargo test -p graphrefly-core --features loom-checked`. - - Verify `#![forbid(unsafe_code)]` is preserved at every crate root. - -When the diff touches binding crates (also via `mise run run-logged -- …` so a -slow build is observable, not a false-hang): -- `cd crates/graphrefly-bindings-js && pnpm build` (napi-rs) -- `cd crates/graphrefly-bindings-py && maturin develop` (pyo3 — only when verifying py bindings, not part of default flow) -- `cd crates/graphrefly-bindings-wasm && wasm-pack build` (wasm-bindgen) - -Do NOT use `cargo build --workspace` / `cargo nextest run --workspace` unless all binding toolchains are installed; the workspace excludes them from default-members for this reason. - -If a failure is related to an implementation design question, **HALT** and raise it to the user before fixing. - ---- - -## Phase 4: Documentation Updates - -**Skip this phase if `--skip-docs` was passed.** +**TypeScript (`@graphrefly/ts`):** +1. `pnpm --filter @graphrefly/ts test` (vitest) — all pass. +2. `pnpm run lint` (biome + layer-boundary + typecheck gates); `pnpm run lint:fix` to auto-format. +3. `pnpm run build` (tsup ESM/CJS/DTS) when public API changed. -**Authoritative checklist:** follow **`docs/docs-guidance.md`** end-to-end (authority order, Tier 0–5, JSDoc tag table, `gen-api-docs.mjs` REGISTRY, `docs:gen` / `docs:gen:check`, `sync-docs`, when to edit which file). +**Sibling packages (only if touched):** run that package's own test/lint/type gate in its repo (`~/src/graphrefly-py` / `~/src/graphrefly-rs`) following its local conventions. The cross-language contract is behavioral conformance (D24) — a substrate behavior change should drive its `spec/conformance.jsonl` arm green per runtime. -Update documentation when behavior or public API changed: +**jsonl touched (`~/src/graphrefly`):** `node ~/src/graphrefly/dashboard/build.mjs --check` — the consistency gate (non-zero on broken links / orphans). -- **`docs/docs-guidance.md`** — if documentation *conventions* or generator workflow change, update this file so `/qa` and contributors stay aligned -- **`~/src/graphrefly/GRAPHREFLY-SPEC.md`** — only if the **spec** itself is intentionally revised (rare; use semver rules in spec §8) -- **`docs/implementation-plan.md`** — **canonical pre-1.0 sequencer.** When a phase / sub-section item lands, mark it ✅ in the matching Phase 11–16 entry (e.g. "11.1 EC2/EC7 — bridge `value == null` → `=== undefined` ✅ landed") and tag with the date. When all items in a sub-section land, mark the sub-section ✅. When a **whole Phase** lands (every sub-section ✅, no in-flight WAIT/POST-1.0 carries that still need this phase's body to be readable), **archive it**: append a JSONL line per sub-section to the matching `archive/roadmap/phase--*.jsonl` and replace the in-file body with a 2–4-line summary + archive pointer (id, file). Single residual follow-ups move to `optimizations.md` with a back-link. See `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/implementation-plan.md`". New deferred items surfaced by /qa go to `optimizations.md` (line-item state) and may also need a sub-bullet in the matching implementation-plan phase if they reshape its scope. -- **`docs/optimizations.md`** — add **new open decisions** under "Active work items" (line-item state for the new carry; cross-link from the matching implementation-plan.md phase if relevant). **Then actively sweep:** scan for any fully-resolved items (all sub-tasks DONE, no remaining TODOs) and archive them to `archive/optimizations/resolved-decisions.jsonl` per `docs/docs-guidance.md` § "Optimization decision log". Remove archived content from `optimizations.md` — it should contain only active/open items, anti-patterns, and deferred follow-ups. -- **Structured JSDoc** on exported public APIs (Tier 1 — parameters, returns, examples per `docs-guidance`; source of truth for generated API pages). `@example` imports must use the correct subpath for the symbol's tier (universal / `/node` / `/browser`). -- **New public symbols** — barrel export in the pure-ts package (`packages/pure-ts/src//index.ts`) + **`website/scripts/gen-api-docs.mjs` REGISTRY** entry, then `pnpm --filter @graphrefly/docs-site docs:gen` (or `docs:gen:check` in CI). If the symbol introduced a new subpath, the post-Phase-13.9.A 4-file rule applies: update `packages/pure-ts/tsup.config.ts` (`ENTRY_POINTS` + `nodeOnlyEntries` when node-only) AND `packages/pure-ts/package.json` `exports`, AND mirror in the root shim's `tsup.config.ts` `ENTRY_POINTS` + root `package.json` `exports`, AND create the matching one-liner shim source at root `src/.ts` (`export * from "@graphrefly/pure-ts/"`). -- **`docs/test-guidance.md`** — if new test patterns are established -- **`docs/roadmap.md`** — **vision / wave context only** per 2026-04-30 migration. Do NOT track item-level state here; that lives in `implementation-plan.md`. Only edit the roadmap when the strategic frame shifts (a wave completes, a positioning lock changes). When a wave or Phase 7.x / 8.x section is fully done, archive its body to `archive/roadmap/*.jsonl` and leave a one-line pointer per `docs/docs-guidance.md` § "Roadmap archive — Workflow for `docs/roadmap.md`". Most /qa cycles will not touch roadmap.md at all. -- **`CLAUDE.md`** — only if fundamental workflow/commands changed +**TLA+ touched:** re-run the affected model (`cd ~/src/graphrefly/formal && java -cp tlc2.TLC -config .cfg `); for a new invariant, mutation-verify it is load-bearing (break the guard → confirm it trips) before claiming it. -Do **not** hand-edit **`website/src/content/docs/api/*.md`** — regenerate from JSDoc via `docs:gen` per **`docs/docs-guidance.md`**. +**Long / heavy commands** (full test sweeps, Rust gates, TLC): run them so they're observably-finishing, not a false-hang — prefer a run-logged wrapper + monitor a guaranteed DONE sentinel (never a guessed progress string, never `sleep`-poll). If this QA runs in a spawned subagent, run such commands **synchronously** (wait for the sentinel) or tear them down (kill by process group) **before returning** — never leak a live background process as a stale "running" entry. (memories `feedback_long_command_observation`, `feedback_subagent_bg_hygiene`, `feedback_no_chained_background_cargo`.) -### Rust-port QA additions to Phase 4 +If a failure exposes a design question, **HALT** and raise it before fixing. -When the diff is in `~/src/graphrefly-rs/`, also update these (in addition to or instead of the TS-side docs above): - -- **`docs/implementation-plan-13.6-canonical-spec.md`** (graphrefly-ts) — only if QA surfaced a canonical-spec ambiguity that needs to be tightened. Rare; this is the post-Phase 13.6.A locked authority. If touched, also coordinate with the spec-amendment workflow per spec §8 semver rules. - -- **`docs/implementation-plan-13.6-flowcharts.md`** (graphrefly-ts) — only if a new internal method / process / property was added that the flowcharts should visualize, OR if a 🟥 RED node (current-code-vs-canonical drift) was resolved by the slice (turn it into a non-red node). Add the rule-ID cross-reference for any new flowchart node. - -- **`~/src/graphrefly-rs/docs/migration-status.md`** — **always update on Rust-port QA pass.** Reflect: - - Test count post-QA (per-file breakdown helps future readers) - - Audit fixes that landed (F1, F2, etc. style, mirroring the Slice A+B closing template) - - Cross-link new entries in `porting-deferred.md` from the "Carried forward" pointer - - If a milestone closes during QA, add the `## M — closed YYYY-MM-DD` section per the established template - - clippy / fmt / `#![forbid(unsafe_code)]` status - -- **`~/src/graphrefly-rs/docs/porting-deferred.md`** — **always update on Rust-port QA pass.** Reflect: - - **NEW deferred concerns** surfaced by QA but NOT fixed in this slice — add to the appropriate section ("Performance — §10 simplifications deferred", "v1 dispatcher limitations", "Spec divergences acknowledged in v1", or "Phase 13.8 carry-forward follow-ups"). Each entry needs **what / why-deferred / source** triple. - - **CLOSED concerns** — if QA fixes resolve a previously-deferred entry, move that entry to the "Audit fixes landed in Slice X" section (or the closing section in `migration-status.md`) and remove from the active deferred list. - - **Resolved Open Questions** — if a Part-6 SESSION question got resolved during the slice (via design call or impl), update the entry with `~~strikethrough~~` + the resolution note + the date (mirror the Phase 14 header note pattern). +--- -- **TS-side `docs/optimizations.md`** — only if QA surfaced a CROSS-LANGUAGE design question that needs to be tracked alongside TS / PY work. Rust-only deferrals belong in `porting-deferred.md`, not `optimizations.md`. +## Phase 4: Documentation Updates -- **TS-side `docs/implementation-plan.md`** — only if a Phase 13.7 / 13.8 / similar Rust-port-tracking sub-item closes; mark ✅ inline with date. The Rust port does NOT add new Phase 11–16 sub-bullets here; `migration-status.md` is the canonical Rust tracker. +**Skip if `--skip-docs` was passed.** Update only what behavior/API actually changed; clean-slate docs are jsonl (single source of truth): -- **Rustdoc on public API surface** — every new `pub fn` / `pub struct` / `pub enum` in `graphrefly-core` (and binding crates) needs a doc comment with at minimum: behavior summary, `# Panics` (if applicable), and a cross-reference to the canonical spec rule (`R`) when the API encodes a spec invariant. Generated via `cargo doc -p graphrefly-core` (lands later in CI; smoke-check locally). +- **`spec/rules.jsonl`** — only if the protocol itself was intentionally revised → that is a `/spec-amend`, not a casual edit (amend rules + `formal/*.tla` + `spec/conformance.jsonl` together). Flip a conformance-backed rule `draft → active` once its scenario is green on the reference arm + formal lands (cite the precedent). +- **`decisions/decisions.jsonl`** — a new architectural lock surfaced by QA → `/design-review` → user approval → append a `D#` (update the DS-1 `locks` in `sessions/sessions.jsonl`). +- **`spec/conformance.jsonl`** — flip `runtimes.` → `pass` when an arm lands green; add a new C-* scenario for a new behavioral rule; keep `covers` ↔ rule `covers_by` bidirectionally consistent. +- **`plan/phases.jsonl`** — update the CSP-* phase `status`/`note` the change advances. +- **`plan/backlog.jsonl`** — add new deferred items (B# + concrete trigger) surfaced by QA. +- **`plan/antipatterns.jsonl`** — a recurring anti-pattern (+ a `feedback_*` memory if generalizable). +- **`formal/README.md`** — when a TLA+ module was added/changed (module-map row + a mutation-verified note). +- **Structured JSDoc** on new exported public symbols (`packages/ts/src/`); cite the governing R-id/D# when the API encodes a spec invariant. +- **`guide/guide.jsonl`** (G-test / G-composition / G-docs / G-contribute) — if a new test/composition pattern or doc convention was established. +- **`~/src/graphrefly/CLAUDE.md`** — only if a fundamental workflow/command changed. -- **DO NOT update the legacy multi-file TS spec** (`~/src/graphrefly/GRAPHREFLY-SPEC.md` + `COMPOSITION-GUIDE-*.md`) for Rust-port-only findings. The Rust port targets the canonical-spec doc; the multi-file spec is effectively superseded for Rust purposes per the Phase 13.6.A consolidation. +After any `~/src/graphrefly` jsonl edit, re-run `node ~/src/graphrefly/dashboard/build.mjs --check`. -- **`~/src/graphrefly-ts/packages/parity-tests/scenarios//`** (TS-side) — when the Rust slice closes (or partially fills) a milestone in the `packages/parity-tests/README.md` schedule (M1 dispatcher, M2 Slice E Graph, M3 operators, M4 storage, M5 structures), QA should verify the slice author added corresponding `describe.each(impls)` scenarios, OR raise it as a missing-coverage finding. The structural parameterization is in place; the `rustImpl` arm currently exports `null` (activates when `@graphrefly/native` publishes), so new scenarios run against `pureTsImpl` only at the moment but provide cross-impl coverage automatically once `rustImpl` flips. Internal-refactor slices (under unchanged public API, e.g., a §10 perf simplification) don't need parity-test additions. +When done, briefly list files changed + new exports, the fixes applied vs deferred (with B# pointers), and the gate results. diff --git a/packages/ts/src/__tests__/rewire.test.ts b/packages/ts/src/__tests__/rewire.test.ts index 286b6c82..1d982ae7 100644 --- a/packages/ts/src/__tests__/rewire.test.ts +++ b/packages/ts/src/__tests__/rewire.test.ts @@ -17,6 +17,7 @@ function collect(n: Node) { return { msgs, unsub }; } const num = (ctx: Ctx, i: number): number => ctx.depRecords[i].latest as number; +const types = (m: Message[]) => m.map((x) => x[0]); describe("rewire — Q-semantics (R-rewire / D42)", () => { it("addDep wires a cached dep via push-on-subscribe and recomputes", () => { @@ -190,3 +191,67 @@ describe("rewire — rejects (R-rewire / D42)", () => { expect(() => collect(dh as Node)).toThrow(/mid-fn|feedback/); }); }); + +describe("rewire — QA fixes (atomic settle, DIRTY-before-DATA, pause/batch-safe)", () => { + it("setDeps adding ≥2 cached deps settles ATOMICALLY — fn fires once, never on a partial view (P2)", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 10 }); + const c = node([], null, { initial: 100 }); + let runs = 0; + const sawPartial: boolean[] = []; + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + collect(d); + runs = 0; + + const sum3 = (ctx: Ctx) => { + runs++; + // a SENTINEL added dep at invocation = the fn fired on a partial view (the bug) + sawPartial.push( + ctx.depRecords[1].latest === undefined || ctx.depRecords[2].latest === undefined, + ); + ctx.down([["DATA", num(ctx, 0) + num(ctx, 1) + num(ctx, 2)]]); + }; + d.setDeps([a, b, c], sum3); // add b AND c (both cached) in one rewire + expect(runs).toBe(1); // ONE atomic settle, not one fire per added dep + expect(sawPartial).toEqual([false]); // never fired with an added dep still SENTINEL + expect(d.cache).toBe(111); // 1 + 10 + 100 + }); + + it("a rewire-triggered settle emits DIRTY before DATA downstream (D1 / R-dirty-before-data)", () => { + const a = node([], null, { initial: 1 }); + const b = node([], null, { initial: 100 }); + const d = node([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + const { msgs } = collect(d); + msgs.length = 0; // isolate the rewire wave + + d.addDep(b, (ctx) => ctx.down([["DATA", num(ctx, 0) + num(ctx, 1)]])); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // two-phase, glitch-free + expect(msgs.at(-1)).toEqual(["DATA", 101]); + }); + + it("removeDep of the sole dirty dep to zero deps un-dirties downstream via RESOLVED (Q6 / P1)", () => { + let cctx: Ctx | null = null; + const a = node([], null, { initial: 1 }); + const c = node( + [a], + (ctx: Ctx) => { + cctx = ctx; // async leg: defer the emit + }, + { pool: "async" }, + ); + const d = node([c], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + const { msgs } = collect(d); + (cctx as Ctx).down([["DATA", 5]]); // c settles once → d caches 5 + expect(d.cache).toBe(5); + + msgs.length = 0; + a.down([["DATA", 2]]); // a→c DIRTY/DATA; c (async) defers → c emits DIRTY to d → d dirty + expect(d.status).toBe("dirty"); + expect(types(msgs)).toContain("DIRTY"); + + msgs.length = 0; + d.removeDep(c, (ctx) => ctx.down([["DATA", num(ctx, 0)]])); // remove sole dirty dep → zero deps + expect(types(msgs)).toEqual(["RESOLVED"]); // un-dirtied downstream (not a stray DATA) + expect(d.cache).toBe(5); // cache preserved (Q7), no recompute + }); +}); diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index cd373ec3..dff5bbbb 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -106,6 +106,8 @@ export class Node { private _insideRunWave = false; /** R-rewire: reentrancy guard for setDeps/addDep/removeDep (one mutation in flight). */ private _inDepMutation = false; + /** R-rewire: a dep-add push during mutation requests ONE atomic two-phase settle after. */ + private _rewireRunPending = false; /** Node's own terminal: undefined = live, true = COMPLETE, else ERROR payload. */ private _terminal: true | unknown | undefined = undefined; private _hasTorndown = false; @@ -322,6 +324,8 @@ export class Node { } this._inDepMutation = true; + this._rewireRunPending = false; + let zeroDepUnDirty = false; try { // fn swap (SD-1): re-register against the same pool. this._handle = this._dispatcher.register(fn, this._pool); @@ -396,23 +400,27 @@ export class Node { } // Q6 auto-settle: removing the sole dirty contributor closes the wave. With deps - // remaining, recompute via the normal gate (equals absorbs a no-change recompute → - // RESOLVED); with zero deps the node is inert (degenerate fn-no-deps) so just - // un-dirty downstream with a RESOLVED. Cache is preserved either way (Q7). + // remaining, request the atomic settle (recompute; equals absorbs a no-change run → + // RESOLVED). With zero deps the node is inert (degenerate fn-no-deps) — just un-dirty + // downstream. Cache is preserved either way (Q7). if (removedDirtyContributor && this._pending === 0 && this._status === "dirty") { - if (newDeps.length > 0) { - this._maybeRun(); - } else { - this._status = this._hasData ? "settled" : "sentinel"; - if (this._emittedDirtyThisWave) { - this._emittedDirtyThisWave = false; - this._emitToSubs(["RESOLVED"]); - } - } + if (newDeps.length > 0) this._rewireRunPending = true; + else zeroDepUnDirty = true; } } finally { this._inDepMutation = false; } + + // Atomic post-mutation settle (outside the reentrancy guard so a fresh wave runs + // normally): ONE two-phase DIRTY→DATA recompute if any added dep delivered data or a + // sole-dirty dep was removed; else the zero-dep un-dirty via _down (pause/batch-safe). + if (this._rewireRunPending) { + this._rewireRunPending = false; + this._settleRewire(); + } else if (zeroDepUnDirty) { + if (this._emittedDirtyThisWave) this._down([["RESOLVED"]]); + else this._status = this._hasData ? "settled" : "sentinel"; + } } // ── activation / deactivation (lazy; R-rom-ram) ── @@ -595,6 +603,14 @@ export class Node { } private _maybeRun(): void { + // R-rewire: an added cached dep's push-on-subscribe lands here mid-mutation. Defer + // the fn-run to ONE atomic two-phase settle after every added dep is wired, so the + // fn never fires on a partially-populated added-dep view (multi-add) — _settleRewire + // drains this flag. + if (this._inDepMutation) { + this._rewireRunPending = true; + return; + } // R-pause-modes (default): while paused, skip dep-driven fn re-execution and // coalesce — fire once with the latest dep values on final-lock RESUME. if (this._pausable === true && this._isPaused()) { @@ -604,6 +620,28 @@ export class Node { this._tryRun(); } + /** + * R-rewire atomic settle (after a rewire that warrants a recompute): emit a proper + * two-phase DIRTY→DATA wave (R-dirty-before-data — a rewire-triggered settle is a wave), + * once, with every added dep already wired. Mirrors _maybeRun/_tryRun's pause + gate + + * pending guards, then injects the phase-1 DIRTY the added dep's [START,DATA] handshake + * did not carry. + */ + private _settleRewire(): void { + if (this._pausable === true && this._isPaused()) { + this._pausedDepWaveOccurred = true; + return; + } + if (this._pending > 0) return; + if (this._handle === null) { + this._passthroughEmit(); + return; + } + if (!this._hasCalledFnOnce && !(this._partial || this._allDepsSettled())) return; + this._markDirty(); // phase 1 (no-op if already dirty, e.g. removeDep auto-settle) + this._runWave(); // phase 2: fn → DATA/RESOLVED + } + private _tryRun(): void { if (this._pending > 0) return; if (this._handle === null) { From 9e3a7c1bec3f9d137ce5008b26858710a73e93c0 Mon Sep 17 00:00:00 2001 From: David Chen Date: Fri, 29 May 2026 11:24:39 -0700 Subject: [PATCH 008/175] feat(ts): enhance graph operators and add async source functionality - Updated `package.json` to include new linting scripts for async checks. - Introduced new operators in `graph/operators.ts` for enhanced functionality, including `distinctUntilChanged`, `filter`, `map`, `scan`, `take`, and `merge`. - Added async source capabilities in `graph/sources.ts`, allowing for timer, interval, promise, and iterable sources. - Implemented comprehensive tests for new operators and sources in `__tests__` directory, ensuring correct behavior and compliance with async boundaries. - Enhanced the `Node` class to support new async behaviors and improved buffering logic during pause states. - Added a script to enforce no raw async usage outside designated boundaries, ensuring code quality and adherence to async design principles. --- package.json | 3 +- packages/ts/src/__tests__/conformance.test.ts | 50 ++++ packages/ts/src/__tests__/operators.test.ts | 57 +++- packages/ts/src/__tests__/sources.test.ts | 175 ++++++++++++ packages/ts/src/graph/graph.ts | 148 ++-------- packages/ts/src/graph/operators.ts | 155 +++++++++++ packages/ts/src/graph/sources.ts | 261 ++++++++++++++++++ packages/ts/src/index.ts | 21 ++ packages/ts/src/node/node.ts | 10 +- scripts/check-no-raw-async.ts | 123 +++++++++ 10 files changed, 868 insertions(+), 135 deletions(-) create mode 100644 packages/ts/src/__tests__/sources.test.ts create mode 100644 packages/ts/src/graph/operators.ts create mode 100644 packages/ts/src/graph/sources.ts create mode 100644 scripts/check-no-raw-async.ts diff --git a/package.json b/package.json index 43a3ae11..0c713076 100644 --- a/package.json +++ b/package.json @@ -563,8 +563,9 @@ "changeset": "changeset", "version-packages": "changeset version", "release": "pnpm run build && changeset publish", - "lint": "biome check . && tsx scripts/check-layer-boundary.ts && tsx scripts/check-typecheck.ts", + "lint": "biome check . && tsx scripts/check-layer-boundary.ts && tsx scripts/check-no-raw-async.ts && tsx scripts/check-typecheck.ts", "lint:layers": "tsx scripts/check-layer-boundary.ts", + "lint:no-raw-async": "tsx scripts/check-no-raw-async.ts", "lint:typecheck": "tsx scripts/check-typecheck.ts", "lint:fix": "biome check --write .", "format": "biome format --write .", diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index 382161a1..516d0d94 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -272,3 +272,53 @@ describe("C-8 intra-graph runtime rewire (R-rewire / D42)", () => { expect(() => d.setDeps([a], id)).toThrow(/terminal/); }); }); + +describe("C-9 pausable:false async source ignores PAUSE (R-pause-modes / R-async-paused / D44)", () => { + it("delivers its async production immediately under PAUSE — never buffers (resolves B20)", () => { + let cctx: Ctx | null = null; + // depless async LEAF source, pausable:false (timer/interval-class). + const s = node( + [], + (ctx: Ctx) => { + cctx = ctx; + }, + { pool: "async", pausable: false }, + ); + const { msgs } = collect(s); + expect(cctx).not.toBeNull(); // fn ran on activation, no emit yet + + const L = Symbol("pause"); + s.up([["PAUSE", L]]); // pausable:false ⇒ lockset never consulted + msgs.length = 0; + + (cctx as Ctx).down([["DATA", 42]]); // async production WHILE "paused" → delivered immediately + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // NOT buffered (contrast C-2's compute node) + expect(msgs.at(-1)).toEqual(["DATA", 42]); + expect(s.cache).toBe(42); + }); +}); + +describe("C-10 true-mode async leaf source delivers immediately under PAUSE (R-pause-modes / R-async-paused / D44)", () => { + it("a depless async source's own production is not gated in true (default) mode (B20's twin)", () => { + let cctx: Ctx | null = null; + // depless async LEAF source, pausable:true default (fromPromise/fromAsyncIter-class). + const s = node( + [], + (ctx: Ctx) => { + cctx = ctx; + }, + { pool: "async" }, + ); + const { msgs } = collect(s); + expect(cctx).not.toBeNull(); + + const L = Symbol("pause"); + s.up([["PAUSE", L]]); + msgs.length = 0; + + (cctx as Ctx).down([["DATA", 7]]); // leaf source's OWN production → delivered immediately + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // NOT buffered (a COMPUTE node WOULD buffer — C-2) + expect(msgs.at(-1)).toEqual(["DATA", 7]); + expect(s.cache).toBe(7); + }); +}); diff --git a/packages/ts/src/__tests__/operators.test.ts b/packages/ts/src/__tests__/operators.test.ts index 67429963..0fa1df64 100644 --- a/packages/ts/src/__tests__/operators.test.ts +++ b/packages/ts/src/__tests__/operators.test.ts @@ -1,15 +1,21 @@ import { describe, expect, it } from "vitest"; import type { Message } from "../index.js"; -import { graph } from "../index.js"; +import { distinctUntilChanged, filter, graph, initNode, map, merge, scan, take } from "../index.js"; const data = (msgs: Message[]) => msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); -describe("core operators (node sugar, D6/L1.5)", () => { +// D43: operators are free-standing factory definitions instantiated via the generic +// g.initNode funnel (config folded into the factory; fn params annotated). describe still +// shows the real factory name (D6) — recorded at initNode→_add, node stays thin. +describe("core operators (free-standing factories via g.initNode, D43/D6)", () => { it("map emits fn(value)", () => { const g = graph(); const s = g.state(1); - const m = g.map(s, (n) => n * 2); + const m = g.initNode( + map((n: number) => n * 2), + [s], + ); const msgs: Message[] = []; m.subscribe((x) => msgs.push(x)); expect(m.cache).toBe(2); @@ -21,7 +27,10 @@ describe("core operators (node sugar, D6/L1.5)", () => { it("filter emits only when pred holds", () => { const g = graph(); const s = g.state(2); - const evens = g.filter(s, (n) => n % 2 === 0); + const evens = g.initNode( + filter((n: number) => n % 2 === 0), + [s], + ); const msgs: Message[] = []; evens.subscribe((x) => msgs.push(x)); s.set(3); // filtered out @@ -33,7 +42,10 @@ describe("core operators (node sugar, D6/L1.5)", () => { it("scan accumulates with a seed", () => { const g = graph(); const s = g.state(1); - const sum = g.scan(s, (acc: number, v: number) => acc + v, 0); + const sum = g.initNode( + scan((acc: number, v: number) => acc + v, 0), + [s], + ); const msgs: Message[] = []; sum.subscribe((x) => msgs.push(x)); s.set(2); @@ -44,7 +56,7 @@ describe("core operators (node sugar, D6/L1.5)", () => { it("take emits the first n then COMPLETE (terminal-is-forever)", () => { const g = graph(); const s = g.state(1); - const first2 = g.take(s, 2); + const first2 = g.initNode(take(2), [s]); const msgs: Message[] = []; first2.subscribe((x) => msgs.push(x)); s.set(2); // 2nd value → DATA + COMPLETE (a DIRTY precedes the DATA, two-phase) @@ -57,7 +69,7 @@ describe("core operators (node sugar, D6/L1.5)", () => { it("distinctUntilChanged suppresses repeats", () => { const g = graph(); const s = g.state(1); - const d = g.distinctUntilChanged(s); + const d = g.initNode(distinctUntilChanged(), [s]); const msgs: Message[] = []; d.subscribe((x) => msgs.push(x)); s.set(1); // same → suppressed @@ -71,7 +83,7 @@ describe("core operators (node sugar, D6/L1.5)", () => { const g = graph(); const a = g.state(1); const b = g.state(2); - const m = g.merge([a, b]); + const m = g.initNode(merge(), [a, b]); const vals: unknown[] = []; m.subscribe((msg) => { if (msg[0] === "DATA") vals.push(msg[1]); @@ -87,11 +99,36 @@ describe("core operators (node sugar, D6/L1.5)", () => { it("describe shows the real operator factory name (D6), not 'derived'", () => { const g = graph(); const s = g.state(0, { name: "s" }); - g.map(s, (n) => n + 1, { name: "inc" }); - g.filter(s, (n) => n > 0, { name: "pos" }); + g.initNode( + map((n: number) => n + 1), + [s], + { name: "inc" }, + ); + g.initNode( + filter((n: number) => n > 0), + [s], + { name: "pos" }, + ); const snap = g.describe(); const byId = Object.fromEntries(snap.nodes.map((n) => [n.id, n])); expect(byId.inc.factory).toBe("map"); expect(byId.pos.factory).toBe("filter"); }); + + it("operators work bare (no Graph) via the free initNode", () => { + const g = graph(); + const s = g.state(5); + // Bare-node path (D43): the free initNode builds a working Node with NO graph + // registration (s is created on g, the bare operator just subscribes to it). + const doubled = initNode( + map((n: number) => n * 2), + [s], + ); + const msgs: Message[] = []; + doubled.subscribe((x) => msgs.push(x)); + expect(doubled.cache).toBe(10); + s.set(6); + expect(doubled.cache).toBe(12); + expect(data(msgs)).toEqual([10, 12]); + }); }); diff --git a/packages/ts/src/__tests__/sources.test.ts b/packages/ts/src/__tests__/sources.test.ts new file mode 100644 index 00000000..416691aa --- /dev/null +++ b/packages/ts/src/__tests__/sources.test.ts @@ -0,0 +1,175 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Message } from "../index.js"; +import { + fromAny, + fromAsyncIter, + fromIter, + fromPromise, + graph, + interval, + of, + timer, +} from "../index.js"; + +const data = (msgs: Message[]) => + msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); + +const flush = () => new Promise((r) => setTimeout(r, 0)); + +// D43/D40: async sources are binding-layer producer sugar — depless Operator specs run once on +// activation, schedule their work, and emit later via the captured ctx.down (R-no-raw-async: +// setTimeout/Promise confined here). Instantiated via the generic g.initNode funnel. +describe("timer / interval sources (fake timers, D43)", () => { + beforeEach(() => vi.useFakeTimers()); + afterEach(() => vi.useRealTimers()); + + it("timer one-shot emits DATA(0) then COMPLETE", () => { + const g = graph(); + const n = g.initNode(timer(50), []); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + vi.advanceTimersByTime(50); + expect(data(msgs)).toEqual([0]); + expect(msgs[msgs.length - 1][0]).toBe("COMPLETE"); + expect(n.status).toBe("completed"); + }); + + it("interval emits periodic ticks 0,1,2,…", () => { + const g = graph(); + const n = g.initNode(interval(100), []); + const vals: number[] = []; + const unsub = n.subscribe((m) => { + if (m[0] === "DATA") vals.push(m[1] as number); + }); + vi.advanceTimersByTime(100); // first tick → 0 + vi.advanceTimersByTime(100); // → 1 + vi.advanceTimersByTime(100); // → 2 + expect(vals).toEqual([0, 1, 2]); + unsub(); // deactivate → onDeactivation clears the interval (no leak) + }); + + it("deactivation stops the source — no emit after unsubscribe (cleanup contract)", () => { + const g = graph(); + const n = g.initNode(interval(100), []); + const vals: number[] = []; + const unsub = n.subscribe((m) => { + if (m[0] === "DATA") vals.push(m[1] as number); + }); + vi.advanceTimersByTime(100); // tick 0 + unsub(); // deactivate → ctx.onDeactivation clears the interval (D28) + vi.advanceTimersByTime(500); // no live timer → no further ticks + expect(vals).toEqual([0]); // nothing emitted after deactivation + }); +}); + +describe("promise / iterable / coercion sources (D43)", () => { + it("fromPromise resolves to DATA then COMPLETE", async () => { + const g = graph(); + const n = g.initNode(fromPromise(Promise.resolve(7)), []); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + await flush(); + expect(data(msgs)).toEqual([7]); + expect(msgs[msgs.length - 1][0]).toBe("COMPLETE"); + }); + + it("fromPromise rejects to ERROR", async () => { + const g = graph(); + const boom = new Error("boom"); + const n = g.initNode(fromPromise(Promise.reject(boom)), []); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + await flush(); + const last = msgs[msgs.length - 1]; + expect(last[0]).toBe("ERROR"); + expect((last as ["ERROR", unknown])[1]).toBe(boom); + expect(n.status).toBe("errored"); + }); + + it("fromPromise rejecting with undefined → clean ERROR, never [[ERROR, undefined]] (R-data-payload)", async () => { + const g = graph(); + // Promise.reject(undefined) would otherwise reach ctx.down([[ERROR, undefined]]) and throw + // inside the .then callback (unhandled rejection); the source coerces undefined → Error. + const n = g.initNode(fromPromise(Promise.reject(undefined)), []); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + await flush(); + const last = msgs[msgs.length - 1]; + expect(last[0]).toBe("ERROR"); + expect((last as ["ERROR", unknown])[1]).toBeInstanceOf(Error); // non-SENTINEL payload + expect(n.status).toBe("errored"); + }); + + it("fromAsyncIter emits each value then COMPLETE", async () => { + async function* gen() { + yield 1; + yield 2; + yield 3; + } + const g = graph(); + const n = g.initNode(fromAsyncIter(gen()), []); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + await flush(); + expect(data(msgs)).toEqual([1, 2, 3]); + expect(msgs[msgs.length - 1][0]).toBe("COMPLETE"); + }); + + it("of emits a single value then COMPLETE synchronously", () => { + const g = graph(); + const n = g.initNode(of(42), []); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + expect(data(msgs)).toEqual([42]); + expect(msgs[msgs.length - 1][0]).toBe("COMPLETE"); + }); + + it("fromIter emits each value then COMPLETE synchronously", () => { + const g = graph(); + const n = g.initNode(fromIter([1, 2, 3]), []); + const vals: unknown[] = []; + n.subscribe((m) => { + if (m[0] === "DATA") vals.push(m[1]); + }); + expect(vals).toEqual([1, 2, 3]); + }); + + it("fromAny passes an existing Node through", () => { + const g = graph(); + const s = g.state(1); + expect(fromAny(s)).toBe(s); + }); + + it("fromAny lifts a Promise via fromPromise", async () => { + const n = fromAny(Promise.resolve(9)); + const msgs: Message[] = []; + n.subscribe((x) => msgs.push(x)); + await flush(); + expect(data(msgs)).toEqual([9]); + }); + + it("fromAny expands a sync iterable only with {iter:true}", () => { + const expanded = fromAny([1, 2, 3], { iter: true }); + const vals: unknown[] = []; + expanded.subscribe((m) => { + if (m[0] === "DATA") vals.push(m[1]); + }); + expect(vals).toEqual([1, 2, 3]); + + // default: the array is a single scalar DATA value (of) + const scalar = fromAny([1, 2, 3]); + const got: unknown[] = []; + scalar.subscribe((m) => { + if (m[0] === "DATA") got.push(m[1]); + }); + expect(got).toEqual([[1, 2, 3]]); + }); + + it("describe shows the source factory name (D6)", () => { + const g = graph(); + g.initNode(timer(1000), [], { name: "clock" }); + const snap = g.describe(); + const byId = Object.fromEntries(snap.nodes.map((n) => [n.id, n])); + expect(byId.clock.factory).toBe("timer"); + }); +}); diff --git a/packages/ts/src/graph/graph.ts b/packages/ts/src/graph/graph.ts index 41d02eb0..445bccad 100644 --- a/packages/ts/src/graph/graph.ts +++ b/packages/ts/src/graph/graph.ts @@ -18,6 +18,7 @@ import { Node, type NodeOptions } from "../node/node.js"; import { messageTier } from "../protocol/messages.js"; import type { DescribeEdge, DescribeNode, DescribeOpts, DescribeSnapshot } from "./describe.js"; import type { NodeProfile, ObserveStream, Profile } from "./inspect.js"; +import { initNode, type Operator } from "./operators.js"; /** Map a tuple of Nodes to the tuple of their value types (typed value-level fn args). */ type DepValues[]> = { @@ -207,131 +208,34 @@ export class Graph { this._mounts.push({ at: opts.at, graph: child }); } - // ── operators (node sugar, D6/L1.5; describe shows the real factory name) ── - // Core set per L4-Q7 (main package); time-based + higher-order operators are a - // separate subpackage. Each is value-level over the node primitive; the shared _op - // wrapper carries the D30 throw→ERROR boundary. + // ── operator funnel (D43): instantiate any free-standing Operator (node sugar, D6/L1.5) ── + // g.initNode is the single graph-bound entry for the whole operator/source catalog (D40): + // it delegates to the FREE initNode (graph/operators.ts — the D30 throw→ERROR boundary + + // dispatcher binding live there) and records the operator's REAL factory name in the + // inspection index (_add) so describe shows it (D6/R-describe) while the node stays thin + // (R-node-thin). Operators are free-standing factory definitions (graph/operators.ts, + // graph/sources.ts) usable bare via the free initNode(); this funnel is the inspectable + // path. Replaces the per-operator methods. - private _op( - deps: readonly Node[], - factory: string, - body: (ctx: Ctx) => void, - opts: SugarOpts, - ): Node { - const ctxFn: NodeFn = (ctx: Ctx) => { - try { - body(ctx); - } catch (e) { - ctx.down([["ERROR", e]]); - } - }; - const n = new Node([...deps], ctxFn, this._nodeOpts(opts)); - return this._add(n, factory, deps, opts); - } - - /** map: emit fn(value). */ - map(src: Node, fn: (v: S) => T, opts: SugarOpts = {}): Node { - return this._op( - [src as Node], - "map", - (ctx) => { - ctx.down([["DATA", fn(ctx.depRecords[0].latest as S)]]); - }, - opts, - ); - } - - /** filter: emit value only when pred(value) (else skip the wave). */ - filter(src: Node, pred: (v: S) => boolean, opts: SugarOpts = {}): Node { - return this._op( - [src as Node], - "filter", - (ctx) => { - const v = ctx.depRecords[0].latest as S; - if (pred(v)) ctx.down([["DATA", v]]); - }, - opts, - ); - } - - /** scan: stateful accumulator over the upstream (acc seeded once, kept in ctx.state). */ - scan( - src: Node, - reducer: (acc: T, v: S) => T, - seed: T, - opts: SugarOpts = {}, - ): Node { - return this._op( - [src as Node], - "scan", - (ctx) => { - const acc = ctx.state.get() ?? seed; - const next = reducer(acc, ctx.depRecords[0].latest as S); - ctx.state.set(next); - ctx.down([["DATA", next]]); - }, - opts, - ); - } - - /** take: emit the first n values, then COMPLETE (terminal-is-forever). */ - take(src: Node, n: number, opts: SugarOpts = {}): Node { - return this._op( - [src as Node], - "take", - (ctx) => { - if (n <= 0) { - ctx.down([["COMPLETE"]]); - return; - } - const count = ctx.state.get() ?? 0; - if (count >= n) return; // already satisfied - const v = ctx.depRecords[0].latest as S; - const next = count + 1; - ctx.state.set(next); - ctx.down(next >= n ? [["DATA", v], ["COMPLETE"]] : [["DATA", v]]); - }, - opts, - ); - } - - /** distinctUntilChanged: emit only when the value differs from the previous emit. */ - distinctUntilChanged( - src: Node, - eq: (a: S, b: S) => boolean = Object.is, - opts: SugarOpts = {}, - ): Node { - return this._op( - [src as Node], - "distinctUntilChanged", - (ctx) => { - const v = ctx.depRecords[0].latest as S; - const prev = ctx.state.get<{ v: S }>(); - if (prev && eq(prev.v, v)) return; - ctx.state.set({ v }); - ctx.down([["DATA", v]]); - }, - opts, - ); - } - - /** merge: interleave several sources — emit each DATA from whichever dep fired (partial). */ - merge(srcs: readonly Node[], opts: SugarOpts = {}): Node { - return this._op( - srcs as readonly Node[], - "merge", - (ctx) => { - for (const r of ctx.depRecords) { - if (r.batch && r.batch.length > 0) { - for (const v of r.batch) ctx.down([["DATA", v]]); - } - } - }, - { ...opts, partial: true }, - ); + /** + * Instantiate an operator (or source) factory as a registered graph node. `deps` are + * type-checked against the operator's input element type; the output type flows from the + * operator. A source is a depless operator — pass `[]`. Caller `opts` (name/meta/behavioral + * overrides) win over the operator's baked-in `opts`. + */ + initNode( + op: Operator, + deps: readonly Node[], + opts: SugarOpts = {}, + ): Node { + // Node is invariant (T appears in equals); widen the typed deps to the erased Node + // surface the free initNode / _add accept (same cast the old per-operator methods used). + const erased = deps as readonly Node[]; + const n = initNode(op, erased, this._nodeOpts(opts)); + return this._add(n, op.factory, erased, opts); } - // ── inspection: describe (Slice B); observe/profile land in Slice D ── + // ── inspection: describe / observe / profile (D39) ── /** Static structure snapshot (R-describe / D39). `_prefix` carries the mount path. */ describe(opts: DescribeOpts = {}, _prefix = ""): DescribeSnapshot { diff --git a/packages/ts/src/graph/operators.ts b/packages/ts/src/graph/operators.ts new file mode 100644 index 00000000..ce79d79a --- /dev/null +++ b/packages/ts/src/graph/operators.ts @@ -0,0 +1,155 @@ +/** + * Operators as free-standing factory definitions (D43 / D6 / D40 Catalog-first). + * + * Operators are `node` sugar (D6), NOT verbs (D4) and NEVER in parity (D24). Each is a + * pure {@link Operator} spec — `{factory, body, opts}` — built on the bare `node` primitive + * (NOT the `derived` verb): `body` reads `ctx.depRecords` positionally and emits via + * `ctx.down`. Two ways to instantiate: + * + * - {@link initNode}(op, deps, opts?) — bare, no Graph; returns a working `Node`. + * - `g.initNode(op, deps, opts?)` — graph-bound; the SAME call, plus inspection registration + * (the real factory name lands in the graph's `_entries`, so describe/observe/profile + * show it — D6/R-describe — while the node stays thin, R-node-thin). + * + * The D30 throw→ERROR boundary lives once, in the free {@link initNode} (which `g.initNode` + * funnels through). The pure-ts `operatorOpts/partialOperatorOpts/gatedOperatorOpts` trio is + * NOT ported: its sole job (`describeKind:"derived"`) is dead under D6/D39 (factory = the + * REAL operator name); only the `partial`/`terminalAsRealInput` flags survive, carried + * per-operator in `op.opts`. + * + * Config is folded into the factory (`map(fn)`, `scan(reducer, seed)`, `take(n)`, `merge()`), + * so fn params are annotated where the source type can't be inferred (RxJS-pipe cost); + * dep-type safety is recovered by the typed `g.initNode` overloads. + */ + +import type { Ctx, NodeFn } from "../ctx/types.js"; +import { Node, type NodeOptions } from "../node/node.js"; + +/** + * A free-standing operator definition (D43). `TIn` = the element type each dep delivers + * (the operator reads `ctx.depRecords[i].latest as TIn`); `TOut` = the emitted value type. + * `__in`/`__out` are phantom-only (never assigned) — they let `g.initNode` infer + check the + * dep tuple against the operator without an explicit type argument. + */ +export interface Operator { + /** Real operator name shown in describe (D6/L1.5) — recorded at `_add` by `g.initNode`. */ + readonly factory: string; + /** The ctx-body: reads `ctx.depRecords` positionally, emits via `ctx.down`. */ + readonly body: (ctx: Ctx) => void; + /** Behavioral node options the operator needs (e.g. `partial:true` for combine-family). */ + readonly opts?: Partial>; + /** @internal phantom — never set; carries the dep element type for `g.initNode` inference. */ + readonly __in?: TIn; + /** @internal phantom — never set; carries the output value type. */ + readonly __out?: TOut; +} + +/** + * Instantiate an operator as a bare `Node` — NO Graph required (D43 bare-node path). + * Wraps `op.body` in the D30 value-throw→ERROR boundary; merges `op.opts` with caller `opts` + * (caller wins, so `partial`/`pool`/`equals`/`dispatcher` overrides take effect). The + * dispatcher defaults to the process-global (D26) unless `opts.dispatcher` is passed. + * + * `g.initNode` is the same call PLUS graph registration; graph inspection + * (describe/observe/profile) requires that path — a node built bare here is not registered in + * any graph's index. + */ +export function initNode( + op: Operator, + deps: readonly Node[], + opts: NodeOptions = {}, +): Node { + const body: NodeFn = (ctx) => { + try { + op.body(ctx); + } catch (e) { + ctx.down([["ERROR", e]]); // D30: value-level throw → ERROR + } + }; + return new Node([...deps], body, { ...op.opts, ...opts }); +} + +/** map: emit fn(value). */ +export function map(fn: (v: S) => T): Operator { + return { + factory: "map", + body: (ctx) => { + ctx.down([["DATA", fn(ctx.depRecords[0].latest as S)]]); + }, + }; +} + +/** filter: emit value only when pred(value) (else skip the wave). */ +export function filter(pred: (v: S) => boolean): Operator { + return { + factory: "filter", + body: (ctx) => { + const v = ctx.depRecords[0].latest as S; + if (pred(v)) ctx.down([["DATA", v]]); + }, + }; +} + +/** scan: stateful accumulator over the upstream (acc seeded once, kept in ctx.state). */ +export function scan(reducer: (acc: T, v: S) => T, seed: T): Operator { + return { + factory: "scan", + body: (ctx) => { + const acc = ctx.state.get() ?? seed; + const next = reducer(acc, ctx.depRecords[0].latest as S); + ctx.state.set(next); + ctx.down([["DATA", next]]); + }, + }; +} + +/** take: emit the first n values, then COMPLETE (terminal-is-forever). */ +export function take(n: number): Operator { + return { + factory: "take", + body: (ctx) => { + if (n <= 0) { + ctx.down([["COMPLETE"]]); + return; + } + const count = ctx.state.get() ?? 0; + if (count >= n) return; // already satisfied + const v = ctx.depRecords[0].latest as S; + const next = count + 1; + ctx.state.set(next); + ctx.down(next >= n ? [["DATA", v], ["COMPLETE"]] : [["DATA", v]]); + }, + }; +} + +/** distinctUntilChanged: emit only when the value differs from the previous emit. */ +export function distinctUntilChanged(eq: (a: S, b: S) => boolean = Object.is): Operator { + return { + factory: "distinctUntilChanged", + body: (ctx) => { + const v = ctx.depRecords[0].latest as S; + const prev = ctx.state.get<{ v: S }>(); + if (prev && eq(prev.v, v)) return; + ctx.state.set({ v }); + ctx.down([["DATA", v]]); + }, + }; +} + +/** + * merge: interleave several sources — emit each DATA from whichever dep fired. `partial:true` + * (combine-family): fires on any single dep, not gated on all deps settling. + */ +export function merge(): Operator { + return { + factory: "merge", + opts: { partial: true }, + body: (ctx) => { + for (const r of ctx.depRecords) { + if (r.batch && r.batch.length > 0) { + for (const v of r.batch) ctx.down([["DATA", v]]); + } + } + }, + }; +} diff --git a/packages/ts/src/graph/sources.ts b/packages/ts/src/graph/sources.ts new file mode 100644 index 00000000..98deb462 --- /dev/null +++ b/packages/ts/src/graph/sources.ts @@ -0,0 +1,261 @@ +/** + * Async + sync sources (binding-layer sugar, D43 / D40 / feedback_async_sources_binding_layer). + * + * Sources are depless {@link Operator}`` specs built on the `producer`/`node` path: + * the body runs ONCE on activation (node `_activate`), schedules its work, and emits later via + * the captured `ctx.down` (an external tier-3 emit → the leading DIRTY is synthesized for it, + * R-dirty-before-data). Async lives ONLY here (R-no-raw-async / F-SYNC-CORE): `setTimeout` / + * `Promise` / `for await` are confined to source bodies; the wave core stays sync. Cleanup is + * `ctx.onDeactivation` (D28), NOT a returned object. Per-language (D6/D24), never in parity. + * + * Instantiate via `g.initNode(timer(1000), [])` (graph-bound, inspectable) or + * {@link initNode}(timer(1000), []) (bare). Pool/pause defaults (D43): timer/interval = + * sync + pausable:false (keep producing through PAUSE); fromPromise/fromAsyncIter = async + * (default pausable, so a paused source buffers its late emits — R-async-paused). + */ + +import type { Ctx } from "../ctx/types.js"; +import type { Node, NodeOptions } from "../node/node.js"; +import { initNode, type Operator } from "./operators.js"; + +/** + * Values accepted by {@link fromAny}: an existing Node, a Promise, an (a)sync iterable, or a + * bare scalar. The universal coercion target for higher-order operators (CSP-2.7). + */ +export type NodeInput = Node | PromiseLike | AsyncIterable | Iterable | T; + +/** + * Internal: build a depless source spec. `setup` runs once on activation; its returned fn (if + * any) is registered as the deactivation cleanup (clear timers / abort). `setup` schedules the + * async work and emits via `ctx.down` — synchronously (sync sources) or later (async sources). + */ +function source( + factory: string, + // biome-ignore lint/suspicious/noConfusingVoidType: setup returns a cleanup fn OR nothing — the void arm keeps no-return source bodies (e.g. `of`/`fromIter`) ergonomic, same idiom as EffectFn. + setup: (ctx: Ctx) => void | (() => void), + opts?: Partial>, +): Operator { + return { + factory, + opts, + body: (ctx) => { + const cleanup = setup(ctx); + if (typeof cleanup === "function") ctx.onDeactivation(cleanup); + }, + }; +} + +function isNode(x: unknown): x is Node { + return ( + x != null && + typeof x === "object" && + "cache" in x && + typeof (x as Node).subscribe === "function" + ); +} + +function isThenable(x: unknown): x is PromiseLike { + return x != null && typeof (x as PromiseLike).then === "function"; +} + +/** + * R-data-payload: an ERROR wave must carry a non-SENTINEL payload. A rejection / abort / + * `throw` can legitimately be `undefined` (`Promise.reject(undefined)`, a bare + * `controller.abort()`, `throw undefined`); emitting `[[ERROR, undefined]]` would be rejected + * by the substrate (node `_down`) — and since the source emits from an async callback OUTSIDE + * the synchronous D30 boundary, that throw would surface as an unhandled rejection rather than + * a clean ERROR wave. Coerce `undefined` to a real Error so the source always emits a valid wave. + */ +function errorPayload(reason: unknown): unknown { + return reason === undefined ? new Error("source errored without a reason") : reason; +} + +/** Options shared by the async sources: an optional AbortSignal → ERROR on abort. */ +export interface AsyncSourceOpts { + signal?: AbortSignal; +} + +/** + * Timer source: one-shot (first tick then COMPLETE) or periodic (`{period}` → 0, 1, 2, …). + * sync pool + pausable:false (a timer keeps producing through PAUSE, R-pause-modes). Emits the + * tick counter from 0; deactivation clears the timers. + */ +export function timer(ms: number, opts?: { period?: number }): Operator { + const period = opts?.period; + return source( + "timer", + (ctx) => { + let done = false; + let count = 0; + let t: ReturnType | undefined; + let iv: ReturnType | undefined; + const finish = () => { + if (done) return; + if (period != null) { + ctx.down([["DATA", count++]]); + iv = setInterval(() => { + if (done) return; + ctx.down([["DATA", count++]]); + }, period); + } else { + // One-shot: DATA then COMPLETE in one wave (terminal-is-forever). + done = true; + ctx.down([["DATA", count++], ["COMPLETE"]]); + } + }; + t = setTimeout(finish, ms); + return () => { + done = true; + if (t !== undefined) clearTimeout(t); + if (iv !== undefined) clearInterval(iv); + }; + }, + { pool: "sync", pausable: false }, + ); +} + +/** interval: periodic ticks (0, 1, 2, …), first at `ms`, then every `ms` (RxJS semantics). */ +export function interval(ms: number): Operator { + return timer(ms, { period: ms }); +} + +/** + * Lift a Promise (or thenable) to a single-value stream: one DATA then COMPLETE, or ERROR on + * rejection. async pool (default pausable → a paused source buffers its late emit). Optional + * `signal` aborts to ERROR. + */ +export function fromPromise( + p: Promise | PromiseLike, + opts?: AsyncSourceOpts, +): Operator { + const signal = opts?.signal; + return source( + "fromPromise", + (ctx) => { + let settled = false; + const onAbort = () => { + if (settled) return; + settled = true; + // onAbort only runs from a `signal` listener / the `signal.aborted` guard → signal is defined. + ctx.down([["ERROR", errorPayload((signal as AbortSignal).reason)]]); + }; + if (signal?.aborted) { + onAbort(); + return; + } + signal?.addEventListener("abort", onAbort, { once: true }); + void Promise.resolve(p).then( + (v) => { + if (settled) return; + settled = true; + signal?.removeEventListener("abort", onAbort); + ctx.down([["DATA", v], ["COMPLETE"]]); + }, + (e) => { + if (settled) return; + settled = true; + signal?.removeEventListener("abort", onAbort); + ctx.down([["ERROR", errorPayload(e)]]); + }, + ); + return () => { + settled = true; + signal?.removeEventListener("abort", onAbort); + }; + }, + { pool: "async" }, + ); +} + +/** + * Read an async iterable: each value → DATA; COMPLETE when done; ERROR on failure. async pool. + * Optional `signal` aborts the pump. + */ +export function fromAsyncIter( + iterable: AsyncIterable, + opts?: AsyncSourceOpts, +): Operator { + const outerSignal = opts?.signal; + return source( + "fromAsyncIter", + (ctx) => { + const ac = new AbortController(); + const onOuterAbort = () => ac.abort(outerSignal?.reason); + outerSignal?.addEventListener("abort", onOuterAbort, { once: true }); + let done = false; + const pump = async () => { + try { + for await (const v of iterable) { + if (ac.signal.aborted) break; + ctx.down([["DATA", v]]); + } + if (!ac.signal.aborted) ctx.down([["COMPLETE"]]); + } catch (e) { + if (!ac.signal.aborted) ctx.down([["ERROR", errorPayload(e)]]); + } finally { + done = true; + } + }; + void pump(); + return () => { + if (!done) ac.abort(); + outerSignal?.removeEventListener("abort", onOuterAbort); + }; + }, + { pool: "async" }, + ); +} + +/** of: a single synchronous value — DATA then COMPLETE, emitted on activation. */ +export function of(value: T): Operator { + return source( + "of", + (ctx) => { + ctx.down([["DATA", value], ["COMPLETE"]]); + }, + { pool: "sync" }, + ); +} + +/** fromIter: a sync iterable — each value → DATA (in order), then COMPLETE, on activation. */ +export function fromIter(iterable: Iterable): Operator { + return source( + "fromIter", + (ctx) => { + for (const v of iterable) ctx.down([["DATA", v]]); + ctx.down([["COMPLETE"]]); + }, + { pool: "sync" }, + ); +} + +/** + * Coerce a {@link NodeInput}`` to a `Node` (D43 — coercion prerequisite for the CSP-2.7 + * higher-order operators). An existing Node passes through; a thenable → {@link fromPromise}; + * an async iterable → {@link fromAsyncIter}; with `{iter:true}` a sync iterable → + * {@link fromIter}; everything else → {@link of}. Coerced nodes are bare (use + * `opts.dispatcher` to bind one, default = process-global D26). + */ +export function fromAny( + input: NodeInput, + opts: NodeOptions & { iter?: boolean } = {}, +): Node { + if (isNode(input)) return input as Node; + const { iter, ...nodeOpts } = opts; + if (isThenable(input)) { + return initNode(fromPromise(input as PromiseLike), [], nodeOpts); + } + if (input !== null && input !== undefined) { + const candidate = input as { + [Symbol.asyncIterator]?: unknown; + [Symbol.iterator]?: unknown; + }; + if (typeof candidate[Symbol.asyncIterator] === "function") { + return initNode(fromAsyncIter(input as AsyncIterable), [], nodeOpts); + } + if (iter === true && typeof candidate[Symbol.iterator] === "function") { + return initNode(fromIter(input as Iterable), [], nodeOpts); + } + } + return initNode(of(input as T), [], nodeOpts); +} diff --git a/packages/ts/src/index.ts b/packages/ts/src/index.ts index 363f5476..62947630 100644 --- a/packages/ts/src/index.ts +++ b/packages/ts/src/index.ts @@ -32,5 +32,26 @@ export { type SugarOpts, } from "./graph/graph.js"; export type { NodeProfile, ObserveEvent, ObserveStream, Profile } from "./graph/inspect.js"; +export { + distinctUntilChanged, + filter, + initNode, + map, + merge, + type Operator, + scan, + take, +} from "./graph/operators.js"; +export { + type AsyncSourceOpts, + fromAny, + fromAsyncIter, + fromIter, + fromPromise, + interval, + type NodeInput, + of, + timer, +} from "./graph/sources.js"; export { dynamicNode, Node, type NodeOptions, node, type Status } from "./node/node.js"; export * from "./protocol/messages.js"; diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index dff5bbbb..673d9b08 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -975,10 +975,16 @@ export class Node { /** Should an outgoing settle slice be deferred into the pause buffer? */ private _shouldBufferOnPause(): boolean { + // D44: pausable mode is the OUTER gate over R-async-paused buffering. + // false: ignore PAUSE/RESUME ENTIRELY — never buffer, keep producing (R-pause-modes; resolves B20). + if (this._pausable === false) return false; if (!this._isPaused()) return false; + // resumeAll: production-gating — buffer the node's own (sync/async) settle slice too. if (this._pausable === "resumeAll") return true; - // R-async-paused / DR-3: a late emit (outside runWave) from an async-pool node buffers. - if (!this._insideRunWave && this._isAsyncPool()) return true; + // true (default): PAUSE gates recomputation/propagation, NOT a leaf source's own production. + // An async COMPUTE node's (deps>0) in-flight result buffers (R-async-paused / C-2); a depless + // async leaf source's own production delivers immediately (R-pause-modes / C-10). + if (!this._insideRunWave && this._isAsyncPool() && this._deps.length > 0) return true; return false; } diff --git a/scripts/check-no-raw-async.ts b/scripts/check-no-raw-async.ts new file mode 100644 index 00000000..37c9ab7d --- /dev/null +++ b/scripts/check-no-raw-async.ts @@ -0,0 +1,123 @@ +/** + * R-no-raw-async enforcement for the clean-slate @graphrefly/ts package (B21 / D43). + * + * Raw async primitives must live ONLY at the sanctioned async boundary: sources + * (`graph/sources.ts`) and the pool/runner layer (R-no-raw-async / F-SYNC-CORE — + * "async boundaries live only in sources and the pool/runner layer"). The sync wave + * core and the rest of the graph layer must stay sync. This catches a raw + * `setTimeout`/`Promise`/`for await`/`async` leaking outside that boundary. + * + * Biome's GritQL can't express this (same reason `check-layer-boundary.ts` is a + * script), and `noRestrictedGlobals` can't cover `new Promise`/`Promise.resolve`/ + * `for await` (not globals) — so the full R-no-raw-async surface lives here, wired + * into `pnpm lint`. + * + * Comment-aware: strips `//` and block comments (newlines preserved) so prose + * mentions ("kicks off async work") don't trip the scan. Test files are exempt. + */ + +import { readdirSync, readFileSync, statSync } from "node:fs"; +import { join, relative, resolve } from "node:path"; + +const ROOT = resolve(import.meta.dirname, ".."); +const SRC = join(ROOT, "packages/ts/src"); + +/** + * Files where raw async IS the sanctioned boundary (R-no-raw-async). Keep this set + * as small as possible — every entry is a hole in the guard. When a real async pool + * (WorkerPool/RemotePool, D20) lands, add the pool/runner file here. + */ +const ALLOW = new Set(["packages/ts/src/graph/sources.ts"]); + +/** Risky code patterns, matched AFTER comment-stripping. */ +const PATTERNS: Array<[RegExp, string]> = [ + [/\bsetTimeout\s*\(/, "setTimeout("], + [/\bsetInterval\s*\(/, "setInterval("], + [/\bsetImmediate\s*\(/, "setImmediate("], + [/\bqueueMicrotask\s*\(/, "queueMicrotask("], + [/\bprocess\s*\.\s*nextTick\b/, "process.nextTick"], + [/\bnew\s+Promise\b/, "new Promise"], + [/\bPromise\s*\.\s*(?:resolve|reject|all|race|allSettled|any)\s*\(/, "Promise.()"], + [/\bfor\s+await\b/, "for await"], + [/\basync\s+(?:function|\*|\(|[A-Za-z_$])/, "async function/method/arrow"], + [/\bawait\s/, "await"], +]; + +/** Strip `//` line comments and `/* *\/` block comments, preserving newlines for line numbers. */ +function stripComments(text: string): string { + let out = ""; + let state: "code" | "line" | "block" = "code"; + for (let i = 0; i < text.length; i++) { + const two = text.slice(i, i + 2); + if (state === "code") { + if (two === "//") { + state = "line"; + i++; + } else if (two === "/*") { + state = "block"; + i++; + } else { + out += text[i]; + } + } else if (state === "line") { + if (text[i] === "\n") { + state = "code"; + out += "\n"; + } + } else { + // block + if (two === "*/") { + state = "code"; + i++; + } else if (text[i] === "\n") { + out += "\n"; + } + } + } + return out; +} + +function walk(dir: string, out: string[]): void { + for (const e of readdirSync(dir, { withFileTypes: true })) { + if (e.name === "node_modules" || e.name === "dist" || e.name === "__tests__") continue; + const p = join(dir, e.name); + if (e.isDirectory()) walk(p, out); + else if (/\.(ts|tsx|mts|cts)$/.test(e.name)) out.push(p); + } +} + +const files: string[] = []; +try { + if (statSync(SRC).isDirectory()) walk(SRC, files); +} catch { + console.error("check-no-raw-async: packages/ts/src not found"); + process.exit(1); +} + +let violations = 0; +for (const fileAbs of files) { + const repoRel = relative(ROOT, fileAbs).split("\\").join("/"); + if (ALLOW.has(repoRel)) continue; + const lines = stripComments(readFileSync(fileAbs, "utf8")).split("\n"); + for (let i = 0; i < lines.length; i++) { + for (const [re, label] of PATTERNS) { + if (re.test(lines[i])) { + violations++; + console.error( + `check-no-raw-async: ${repoRel}:${i + 1} uses raw async \`${label}\` — ` + + "async boundaries live ONLY in sources / the pool-runner layer (R-no-raw-async / F-SYNC-CORE). " + + "Move it into a source (graph/sources.ts), or add this file to the ALLOW set if it IS the pool/runner boundary.", + ); + } + } + } +} + +if (violations > 0) { + console.error(`\ncheck-no-raw-async: ${violations} violation(s).`); + process.exit(1); +} +console.log( + `check-no-raw-async: ${files.length} files checked, no raw async outside the sanctioned boundary ` + + `(${ALLOW.size} allowlisted).`, +); From 7434e69ca81182c8eb5aeddfece2c5cfdeb4c534 Mon Sep 17 00:00:00 2001 From: David Chen Date: Sat, 30 May 2026 00:20:57 -0700 Subject: [PATCH 009/175] refactor(tests): update dynamicNode and conformance tests for DATA handling - Renamed test descriptions in `batch-dynamic.test.ts` to clarify behavior regarding unread dependencies and their emissions as DATA occurrences. - Enhanced assertions in `conformance.test.ts` to reflect the new behavior of occurrences remaining as DATA, ensuring compliance with the updated handling of undirty RESOLVED states. - Introduced new tests in `dispatcher.test.ts` to validate the proper functioning of the Dispatcher, including handle registration and unregistration, ensuring no memory leaks occur during rewire operations. - Updated lifecycle tests to confirm that terminal states correctly prevent further emissions after self-emit attempts, maintaining cache integrity. - Added QA tests to ensure that the system rejects invalid DATA and RESOLVED combinations, reinforcing tier-3 exclusivity rules. --- .../ts/src/__tests__/batch-dynamic.test.ts | 14 +- packages/ts/src/__tests__/conformance.test.ts | 209 +++++++++++++++++- packages/ts/src/__tests__/core.test.ts | 11 +- packages/ts/src/__tests__/dispatcher.test.ts | 91 ++++++++ packages/ts/src/__tests__/lifecycle.test.ts | 27 +++ packages/ts/src/__tests__/qa-fixes.test.ts | 16 +- packages/ts/src/__tests__/rewire.test.ts | 27 ++- packages/ts/src/ctx/types.ts | 4 +- packages/ts/src/dispatcher/index.ts | 68 ++++-- packages/ts/src/graph/graph.ts | 23 +- packages/ts/src/graph/operators.ts | 2 +- packages/ts/src/node/node.ts | 135 ++++++++--- 12 files changed, 539 insertions(+), 88 deletions(-) create mode 100644 packages/ts/src/__tests__/dispatcher.test.ts diff --git a/packages/ts/src/__tests__/batch-dynamic.test.ts b/packages/ts/src/__tests__/batch-dynamic.test.ts index 11f70961..62853798 100644 --- a/packages/ts/src/__tests__/batch-dynamic.test.ts +++ b/packages/ts/src/__tests__/batch-dynamic.test.ts @@ -71,8 +71,8 @@ describe("batch (R-batch-coalesce / D12)", () => { }); }); -describe("dynamicNode (R-dynamic-node / D35)", () => { - it("reads a selected dep; an unread dep's change is absorbed (no downstream DATA)", () => { +describe("dynamicNode (R-dynamic-node / D35; D49 — no equals-absorption)", () => { + it("reads a selected dep; an unread dep's change re-emits the value as DATA (D49)", () => { const sel = node<"a" | "b">([], null, { initial: "a" }); const a = node([], null, { initial: 100 }); const b = node([], null, { initial: 200 }); @@ -85,13 +85,15 @@ describe("dynamicNode (R-dynamic-node / D35)", () => { expect(router.cache).toBe(100); // sel="a" -> reads a msgs.length = 0; - // change the UNREAD dep b -> fn fires, output unchanged (still 100) -> equals absorbs + // change the UNREAD dep b -> fn fires, re-emits the (unchanged) value 100. D49 removed + // equals-absorption: an occurrence is always DATA, never collapsed to RESOLVED. Pair with + // distinctUntilChanged if unread-dep silence is required (dedup is opt-in). b.down([["DATA", 999]]); expect(router.cache).toBe(100); - expect(types(msgs)).not.toContain("DATA"); - expect(types(msgs)).toContain("RESOLVED"); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(msgs).toContainEqual(["DATA", 100]); - // change the READ dep a -> downstream DATA + // change the READ dep a -> recompute with the new value msgs.length = 0; a.down([["DATA", 111]]); expect(router.cache).toBe(111); diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index 516d0d94..5e726dc3 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -10,9 +10,11 @@ import { describe, expect, it } from "vitest"; import type { Ctx, Message } from "../index.js"; -import { graph, node } from "../index.js"; +import { distinctUntilChanged, filter, fromIter, graph, node, take } from "../index.js"; const types = (msgs: Message[]) => msgs.map((m) => m[0]); +const data = (msgs: Message[]) => + msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); function collect(n: { subscribe(s: (m: Message) => void): () => void }) { const msgs: Message[] = []; const unsub = n.subscribe((m) => msgs.push(m)); @@ -322,3 +324,208 @@ describe("C-10 true-mode async leaf source delivers immediately under PAUSE (R-p expect(s.cache).toBe(7); }); }); + +// C-12 (D49 / R-resolved-undirty, supersedes D15/R-equals): every value-occurrence is DATA +// (no auto-equals-substitution); RESOLVED is the substrate-SYNTHESIZED undirty-only signal; +// dedup is opt-in at the operator layer. Folds the former _probe.test.ts probes (the +// fromIter([1,1,1]) / take(3) / state.set-same cases that surfaced D49). +describe("C-12 occurrences stay DATA; RESOLVED is undirty-only (R-resolved-undirty / D49)", () => { + it("(a) a repeated value is N distinct DATA occurrences, never collapsed to RESOLVED", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 1, 1]), []); + const { msgs } = collect(src); + expect(types(msgs)).toEqual(["START", "DATA", "DATA", "DATA", "COMPLETE"]); + expect(data(msgs)).toEqual([1, 1, 1]); // not [1] — the pre-D49 equals-absorption bug + }); + + it("(b) take(3) counts occurrences, not distinct values → [1,1,1]", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 1, 1]), []); + const t = g.initNode(take(3), [src]); + const { msgs } = collect(t); + expect(data(msgs)).toEqual([1, 1, 1]); + expect(t.status).toBe("completed"); + }); + + it("(c) filter-reject: the substrate synthesizes one undirty RESOLVED — no DATA, no wedge", () => { + const g = graph(); + const s = g.state(50); + const f = g.initNode( + filter((n: number) => n >= 100), + [s], + ); + const { msgs } = collect(f); + // activation: 50 rejected, f never produced — no DIRTY on activation, no synth + expect(f.status).toBe("sentinel"); + expect(f.cache).toBeUndefined(); + msgs.length = 0; + + s.set(60); // rejected: DIRTY'd but no value → substrate-synthesized undirty RESOLVED + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); // un-dirtied, no DATA, not wedged + expect(f.status).toBe("sentinel"); // never valued + msgs.length = 0; + + s.set(150); // accepted → real DATA occurrence + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + expect(f.cache).toBe(150); + msgs.length = 0; + + s.set(70); // rejected, but f now carries 150 → undirty RESOLVED, status resolved + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); + expect(f.status).toBe("resolved"); + expect(f.cache).toBe(150); // cache preserved across the undirty wave + }); + + it("(c') a downstream recompute un-dirties as DATA (occurrence), never wedges", () => { + const g = graph(); + const s = g.state(100); // accepted by the filter + const f = g.initNode( + filter((n: number) => n >= 100), + [s], + ); + const d = g.derived([f], (v: number) => v * 2); + const { msgs } = collect(d); + expect(d.cache).toBe(200); // f=100 → d=200 + msgs.length = 0; + + s.set(50); // rejected by f → f synthesizes RESOLVED → d clears + recomputes f's cached 100 + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // d un-dirtied via a (same-value) DATA, no wedge + expect(d.cache).toBe(200); + }); + + it("(d) distinctUntilChanged is the OPT-IN dedup (operator's job, not substrate)", () => { + const g = graph(); + const s = g.state(1); + const duc = g.initNode(distinctUntilChanged(), [s]); + const { msgs } = collect(duc); + s.set(1); // dup → operator returns without emitting → substrate synthesizes RESOLVED + s.set(2); + s.set(2); // dup → suppressed + s.set(3); + expect(data(msgs)).toEqual([1, 2, 3]); + }); + + it("a state node re-set to the same value emits DATA, not RESOLVED (no substrate equals)", () => { + const g = graph(); + const s = g.state(1); + const { msgs } = collect(s); + msgs.length = 0; + s.set(1); // same value + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // occurrence, NOT equals-absorbed + expect(s.cache).toBe(1); + msgs.length = 0; + s.set(2); + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); + }); + + it("the tier-3 exclusivity guard stays: a wave cannot mix DATA and RESOLVED", () => { + const s = node([], null, { initial: 1 }); + collect(s); + expect(() => s.down([["DATA", 2], ["RESOLVED"]])).toThrow(/tier-3 exclusivity/); + }); +}); + +describe("C-13 INVALIDATE arriving at a paused compute node (R-paused-invalidate / D50)", () => { + it("(a) a sole-dep INVALIDATE supersedes the buffered paused dep-wave → no recompute on RESUME", () => { + let runs = 0; + const d1 = node([], null, { initial: 0 }); + const n = node([d1], (ctx: Ctx) => { + runs++; + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 1]]); // non-guarding + }); + const { msgs } = collect(n); + expect(n.cache).toBe(1); + runs = 0; + msgs.length = 0; + + const L = Symbol("p"); + n.up([["PAUSE", L]]); + d1.down([["DATA", 5]]); // buffered while paused (no recompute) + expect(runs).toBe(0); + d1.down([["INVALIDATE"]]); // supersedes the buffered wave → cancels the paused recompute + n.up([["RESUME", L]]); // RESUME must NOT recompute against the SENTINEL dep + + expect(runs).toBe(0); // the superseded paused dep-wave does not recompute (D50) + expect(n.cache).toBeUndefined(); // stays SENTINEL (own INVALIDATE) — no garbage recompute + expect(msgs.some((m) => m[0] === "DATA")).toBe(false); // no spurious recompute DATA after INVALIDATE + }); + + it("(b) a DATA after the INVALIDATE re-arms the buffer → RESUME recomputes with the new value", () => { + let runs = 0; + const d1 = node([], null, { initial: 0 }); + const n = node([d1], (ctx: Ctx) => { + runs++; + ctx.down([["DATA", (ctx.depRecords[0].latest as number) + 100]]); + }); + collect(n); + runs = 0; + + const L = Symbol("p"); + n.up([["PAUSE", L]]); + d1.down([["DATA", 5]]); // buffered + d1.down([["INVALIDATE"]]); // supersede v1 + d1.down([["DATA", 7]]); // re-arm with v2 + n.up([["RESUME", L]]); + + expect(runs).toBe(1); // recomputes once with the re-armed value + expect(n.cache).toBe(107); // 7 + 100 (v2, not the superseded v1) + }); + + it("(c) a multi-dep INVALIDATE does NOT cancel a surviving dep's buffered update", () => { + let runs = 0; + const d1 = node([], null, { initial: 0 }); + const d2 = node([], null, { initial: 0 }); + const n = node([d1, d2], (ctx: Ctx) => { + runs++; + const a = (ctx.depRecords[0].latest as number | undefined) ?? 0; // guards D1 SENTINEL + const b = ctx.depRecords[1].latest as number; + ctx.down([["DATA", a + b]]); + }); + collect(n); + runs = 0; + + const L = Symbol("p"); + n.up([["PAUSE", L]]); + d1.down([["DATA", 5]]); // buffered + d2.down([["DATA", 9]]); // buffered (the survivor) + d1.down([["INVALIDATE"]]); // D1 superseded; D2's buffered wave survives + n.up([["RESUME", L]]); + + expect(runs).toBe(1); // still recomputes for the surviving dep (no lost update) + expect(n.cache).toBe(9); // D1=SENTINEL→0, D2=9 + }); +}); + +describe("C-14 cleanup hooks are per-run (cleared + re-registered each fn run) (R-cleanup-hooks / D28)", () => { + it("after K runs, onInvalidate + onDeactivation fire ONCE (the latest run's), not K times", () => { + let flush = 0; + let cleanup = 0; + const s = node([], null, { initial: 0 }); + const d = node([s], (ctx: Ctx) => { + ctx.onInvalidate(() => flush++); + ctx.onDeactivation(() => cleanup++); + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + const { unsub } = collect(d); // run 1 (activation, s=0) + s.down([["DATA", 1]]); // run 2 — re-registers (prior run's hooks cleared) + s.down([["DATA", 2]]); // run 3 — re-registers (D has now run 3×) + + s.down([["INVALIDATE"]]); // fires onInvalidate ONCE (run-3's), not 3× (the accumulation bug) + expect(flush).toBe(1); + + unsub(); // D deactivates → fires onDeactivation ONCE (run-3's), not 3× + expect(cleanup).toBe(1); + }); + + it("a single-run node keeps its one registration (no re-run → no clear)", () => { + let cleanup = 0; + const s = node([], null, { initial: 5 }); + const d = node([s], (ctx: Ctx) => { + ctx.onDeactivation(() => cleanup++); + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + const { unsub } = collect(d); // run 1 only + unsub(); + expect(cleanup).toBe(1); + }); +}); diff --git a/packages/ts/src/__tests__/core.test.ts b/packages/ts/src/__tests__/core.test.ts index e5b384f4..e5afaf60 100644 --- a/packages/ts/src/__tests__/core.test.ts +++ b/packages/ts/src/__tests__/core.test.ts @@ -45,13 +45,13 @@ describe("state node (manual source)", () => { }); }); -describe("equals -> RESOLVED (R-equals)", () => { - it("re-emitting the same value yields RESOLVED, not DATA; cache unchanged", () => { +describe("occurrences stay DATA (R-resolved-undirty / D49, supersedes R-equals)", () => { + it("re-emitting the same value yields DATA, not RESOLVED (no equals-substitution)", () => { const s = node([], null, { initial: 5 }); const { msgs } = collect(s); msgs.length = 0; - s.down([["DATA", 5]]); - expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); + s.down([["DATA", 5]]); // same value is still a distinct occurrence + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); expect(s.cache).toBe(5); }); @@ -64,7 +64,7 @@ describe("equals -> RESOLVED (R-equals)", () => { expect(s.cache).toBe(6); }); - it("does NOT substitute on a multi-DATA wave (R-equals exclusivity)", () => { + it("a multi-DATA wave passes every occurrence as DATA", () => { const s = node([], null, { initial: 5 }); const { msgs } = collect(s); msgs.length = 0; @@ -72,7 +72,6 @@ describe("equals -> RESOLVED (R-equals)", () => { ["DATA", 5], ["DATA", 5], ]); - // dataCount > 1 => no equals substitution; both pass as DATA. expect(types(msgs)).toEqual(["DIRTY", "DATA", "DATA"]); }); }); diff --git a/packages/ts/src/__tests__/dispatcher.test.ts b/packages/ts/src/__tests__/dispatcher.test.ts new file mode 100644 index 00000000..bd392ec7 --- /dev/null +++ b/packages/ts/src/__tests__/dispatcher.test.ts @@ -0,0 +1,91 @@ +import { describe, expect, it } from "vitest"; +import type { Ctx } from "../index.js"; +import { Dispatcher } from "../index.js"; + +// invoke only forwards ctx to the fn; these probe fns ignore it. +const ctx = {} as unknown as Ctx; + +describe("dispatcher handle GC (B15 / R-dispatch-all)", () => { + it("unregister frees the slot — the old handle is dead, invoking it throws", () => { + const d = new Dispatcher(); + let ran = 0; + const h = d.register(() => { + ran++; + }, "sync"); + d.invoke(h, ctx); + expect(ran).toBe(1); + + d.unregister(h); + expect(() => d.invoke(h, ctx)).toThrow(); // freed slot → dead handle, not silently ran + expect(ran).toBe(1); // the dropped fn never runs again + }); + + it("register reuses a freed id → the fn table stays bounded under repeated fn-swap", () => { + const d = new Dispatcher(); + const h1 = d.register(() => {}, "sync"); + d.unregister(h1); + const h2 = d.register(() => {}, "sync"); + expect(h2.handleId).toBe(h1.handleId); // reused the freed slot, did not grow + expect(h2.poolId).toBe(h1.poolId); + + // 100 swaps (register new, free old) stay bounded to peak-live, not +100 (the B15 leak). + let peak = h2.handleId; + let cur = h2; + for (let i = 0; i < 100; i++) { + const next = d.register(() => {}, "sync"); + d.unregister(cur); + peak = Math.max(peak, next.handleId); + cur = next; + } + expect(peak).toBeLessThanOrEqual(h1.handleId + 1); // bounded, not h1.handleId + 100 + }); + + it("unregister is idempotent and pool-scoped", () => { + const d = new Dispatcher(); + let ranAsync = 0; + const hs = d.register(() => {}, "sync"); + const ha = d.register(() => { + ranAsync++; + }, "async"); + d.unregister(hs); + d.unregister(hs); // idempotent → no throw on a double free + d.invoke(ha, ctx); // the async pool is untouched by the sync-pool unregister + expect(ranAsync).toBe(1); + expect(ha.poolId).not.toBe(hs.poolId); + }); + + it("a reused handle id does not inherit the previous tenant's profile stat", () => { + const d = new Dispatcher(); + d.setRecording(true); + const h1 = d.register(() => {}, "sync"); + d.invoke(h1, ctx); + d.invoke(h1, ctx); + expect(d.statFor(h1)?.invokes).toBe(2); + + d.unregister(h1); // clears the stat too + const h2 = d.register(() => {}, "sync"); + expect(h2.handleId).toBe(h1.handleId); // same id reused + expect(d.statFor(h2)?.invokes ?? 0).toBe(0); // fresh counters, not the stale 2 + }); + + it("QA F2.1: unregister clears the stat even when recording is OFF (no later inheritance)", () => { + // The stale-stat trap: recording is OFF at unregister time, so a naive `if (recording)` + // delete would leave the key — then a reused id inherits it once recording resumes. + const d = new Dispatcher(); + d.setRecording(true); + const h1 = d.register(() => {}, "sync"); + d.invoke(h1, ctx); + d.invoke(h1, ctx); + expect(d.statFor(h1)?.invokes).toBe(2); + + d.setRecording(false); // recording paused around the rewire/unregister window + d.unregister(h1); // must STILL drop the stat (unconditional delete, QA F2.1) + const h2 = d.register(() => {}, "sync"); + expect(h2.handleId).toBe(h1.handleId); // free-list reused the id + + d.setRecording(true); // resume — h2 must NOT see h1's stale count of 2 + expect(d.statFor(h2)?.invokes ?? 0).toBe(0); + d.invoke(h2, ctx); + expect(d.statFor(h2)?.invokes).toBe(1); // counts from 0, not from the stale 2 → 3 + }); +}); diff --git a/packages/ts/src/__tests__/lifecycle.test.ts b/packages/ts/src/__tests__/lifecycle.test.ts index ecf2199a..7bcdf963 100644 --- a/packages/ts/src/__tests__/lifecycle.test.ts +++ b/packages/ts/src/__tests__/lifecycle.test.ts @@ -72,6 +72,33 @@ describe("terminal (R-terminal, R-deps-terminal)", () => { expect(s.status).toBe("settled"); expect(msgs).toContainEqual(["DATA", 1]); }); + + it("B30: terminal-is-forever on SELF-emit — set()/down() after terminal is a no-op", () => { + // The upstream path (_receiveFromDep) already drops on terminal; B30 closes the + // self-emit gap in _down (a post-terminal state.set / ctx.down must not resurrect), + // matching the Rust arm's Core::down blanket guard (R-terminal / D17). + const s = node([], null, { initial: 1 }); + const { msgs } = collect(s); + s.down([["COMPLETE"]]); + expect(s.status).toBe("completed"); + msgs.length = 0; + + s.down([["DATA", 42]]); // self-emit in a LATER wave after terminal → no-op + expect(msgs).toEqual([]); // nothing emitted + expect(s.cache).toBe(1); // cache NOT overwritten by 42 + expect(s.status).toBe("completed"); // still terminal + + // also a no-op after ERROR (DATA + INVALIDATE both dropped) + const e = node([], null, { initial: 7 }); + const ce = collect(e); + e.down([["ERROR", new Error("boom")]]); + ce.msgs.length = 0; + e.down([["DATA", 99]]); + e.down([["INVALIDATE"]]); + expect(ce.msgs).toEqual([]); // neither DATA nor INVALIDATE escapes + expect(e.cache).toBe(7); // cache preserved (errored node) + expect(e.status).toBe("errored"); + }); }); describe("INVALIDATE (R-invalidate-idempotent, R-cleanup-hooks)", () => { diff --git a/packages/ts/src/__tests__/qa-fixes.test.ts b/packages/ts/src/__tests__/qa-fixes.test.ts index c6a54f20..0a106c78 100644 --- a/packages/ts/src/__tests__/qa-fixes.test.ts +++ b/packages/ts/src/__tests__/qa-fixes.test.ts @@ -16,7 +16,7 @@ describe("QA fixes", () => { expect(() => s.down([["DATA", undefined as unknown as number]])).toThrow(/non-SENTINEL/); }); - it("EC2: rejects DATA + RESOLVED in one wave (R-equals tier-3 exclusivity)", () => { + it("EC2: rejects DATA + RESOLVED in one wave (R-resolved-undirty tier-3 exclusivity)", () => { const s = node([], null, { initial: 1 }); collect(s); expect(() => s.down([["DATA", 2], ["RESOLVED"]])).toThrow(/tier-3 exclusivity/); @@ -60,4 +60,18 @@ describe("QA fixes", () => { const { msgs } = collect(s); // late subscriber expect(types(msgs)).toEqual(["START"]); // no stale DATA replayed }); + + it("WEDGE: a passthrough over a RESOLVED-only dep balances its DIRTY (R-resolved-undirty / D49)", () => { + // Regression for the D49-widened wedge: a passthrough wire (deps, no fn) that broadcast + // DIRTY then receives an undirty RESOLVED from its dep (no DATA in the batch) must emit a + // balancing RESOLVED, not leave a dangling DIRTY. Under D49 a dep emits a bare RESOLVED on + // every filter-reject / distinctUntilChanged-dup / no-emit fn, so this fires constantly. + const dep = node([], null, { initial: 1 }); + const pass = node([dep], null); // passthrough wire (_handle === null) + const { msgs } = collect(pass); + msgs.length = 0; + dep.down([["RESOLVED"]]); // dep settles with no value → DIRTY then RESOLVED to the passthrough + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); // balanced, not a dangling ["DIRTY"] + expect(pass.status).not.toBe("dirty"); // un-wedged + }); }); diff --git a/packages/ts/src/__tests__/rewire.test.ts b/packages/ts/src/__tests__/rewire.test.ts index 1d982ae7..d1ab06d6 100644 --- a/packages/ts/src/__tests__/rewire.test.ts +++ b/packages/ts/src/__tests__/rewire.test.ts @@ -9,7 +9,7 @@ import { describe, expect, it } from "vitest"; import type { Ctx, Message } from "../index.js"; -import { type Node, node } from "../index.js"; +import { Dispatcher, type Node, node } from "../index.js"; function collect(n: Node) { const msgs: Message[] = []; @@ -254,4 +254,29 @@ describe("rewire — QA fixes (atomic settle, DIRTY-before-DATA, pause/batch-saf expect(types(msgs)).toEqual(["RESOLVED"]); // un-dirtied downstream (not a stray DATA) expect(d.cache).toBe(5); // cache preserved (Q7), no recompute }); + + it("B15: a rewire fn-swap frees the old dispatcher handle (registry stays bounded)", () => { + // Each setDeps/addDep/removeDep re-registers the fn → a new handle. Before B15 the old + // handle leaked (the pool fn-table grew by one per swap), unbounded for a rewire-heavy + // graph (CSP-2.7 *Map). Now _rewire unregisters the old handle, so the table is bounded + // to peak-live size and freed ids are reused. Observed via a probe registration on a + // dedicated dispatcher: after N swaps a fresh register reuses a freed slot (small id), + // not ~N (the leak). Mutation-verified: disabling the unregister in _rewire fails this. + const disp = new Dispatcher(); + const a = node([], null, { initial: 1, dispatcher: disp }); + const id = (ctx: Ctx) => ctx.down([["DATA", num(ctx, 0)]]); + const d = node([a], id, { dispatcher: disp }); + collect(d); + + const N = 50; + for (let i = 0; i < N; i++) { + // idempotent dep-set, but SD-1 still swaps the fn each call → register + free old. + d.setDeps([a], (ctx) => ctx.down([["DATA", num(ctx, 0)]])); + } + expect(d.cache).toBe(1); // still correct after 50 swaps + + // A fresh registration reuses a freed slot → bounded handleId, NOT ~N (the leak). + const probe = disp.register(() => {}, "sync"); + expect(probe.handleId).toBeLessThanOrEqual(2); + }); }); diff --git a/packages/ts/src/ctx/types.ts b/packages/ts/src/ctx/types.ts index 85b4d0b9..5e86a543 100644 --- a/packages/ts/src/ctx/types.ts +++ b/packages/ts/src/ctx/types.ts @@ -54,7 +54,9 @@ export interface Ctx { /** * Read a dep's latest value by index (dynamicNode only, R-dynamic-node / D35). * Present only on dynamicNode fns; all declared deps still participate in wave - * tracking, but an unread dep's change leaves the output unchanged -> equals absorbs. + * tracking. Under D49/R-resolved-undirty an unread dep's change still fires the fn, + * which re-emits its (unchanged) output as a DATA occurrence — there is NO equals- + * absorption; pair with distinctUntilChanged to suppress unchanged re-emits (opt-in). */ track?(depIndex: number): unknown; } diff --git a/packages/ts/src/dispatcher/index.ts b/packages/ts/src/dispatcher/index.ts index 0862545b..a4b437c6 100644 --- a/packages/ts/src/dispatcher/index.ts +++ b/packages/ts/src/dispatcher/index.ts @@ -28,37 +28,45 @@ export interface Handle { export interface Pool { readonly kind: PoolKind; register(fn: NodeFn): number; + /** Free a handle's slot so its fn closure is GC'd and the id is reused (B15). */ + unregister(handleId: number): void; invoke(handleId: number, ctx: Ctx): void; } -class LocalSyncPool implements Pool { - readonly kind = "sync" as const; - private fns: NodeFn[] = []; - register(fn: NodeFn): number { - const id = this.fns.length; - this.fns.push(fn); - return id; - } - invoke(handleId: number, ctx: Ctx): void { - this.fns[handleId](ctx); - } -} - /** - * Structurally identical to LocalSync (R9): the "async" nature lives in the fn body, - * not the pool's invoke. The label informs ctx lifecycle (per-invocation ctx, L3-Q5) - * at the node, not a different dispatch path. + * Array-indexed fn table with a free-list (B15). `register` reuses a freed slot before + * growing the array, so a rewire-heavy graph (fn-swap on every setDeps/addDep/removeDep, + * e.g. CSP-2.7 higher-order *Map operators) keeps the table bounded to its peak live size + * instead of leaking a slot + closure per swap. `unregister` tombstones the slot (drops the + * closure reference for GC) and offers the id for reuse. handleId reuse is safe: the graph is + * a single causal domain (D22) and D37 forbids a fn rewiring its own handle mid-run, so a + * freed id is never re-registered while an invoke of it is in flight. The hot path stays a + * raw array index (F-PERF). */ -class LocalAsyncPool implements Pool { - readonly kind = "async" as const; - private fns: NodeFn[] = []; +class PoolTable implements Pool { + constructor(readonly kind: PoolKind) {} + private fns: (NodeFn | undefined)[] = []; + private free: number[] = []; register(fn: NodeFn): number { + const reused = this.free.pop(); + if (reused !== undefined) { + this.fns[reused] = fn; + return reused; + } const id = this.fns.length; this.fns.push(fn); return id; } + unregister(handleId: number): void { + if (this.fns[handleId] === undefined) return; // already free → idempotent + this.fns[handleId] = undefined; // drop the closure (GC) + this.free.push(handleId); // offer the slot for reuse (bounded growth) + } invoke(handleId: number, ctx: Ctx): void { - this.fns[handleId](ctx); + // A live handle always resolves to a fn; a dead/unregistered id throws (TypeError — + // invoking an unregistered handle is a bug, surfaced loudly). Cast keeps the hot path + // a raw index with no extra branch (F-PERF). + (this.fns[handleId] as NodeFn)(ctx); } } @@ -87,8 +95,8 @@ export class Dispatcher { private _totalInvokes = 0; constructor() { - this.syncPoolId = this.addPool(new LocalSyncPool()); - this.asyncPoolId = this.addPool(new LocalAsyncPool()); + this.syncPoolId = this.addPool(new PoolTable("sync")); + this.asyncPoolId = this.addPool(new PoolTable("async")); } /** Turn the profile recorder on/off (D39). Off = zero overhead on invoke. */ @@ -122,6 +130,22 @@ export class Dispatcher { return { poolId, handleId }; } + /** + * Release a handle (B15): frees the pool slot (closure GC'd, id reusable) and drops any + * accumulated profile stat so a reused id never inherits the previous tenant's counters. + * Called on rewire fn-swap (node._rewire) — the old handle is dropped before the node + * adopts the new one. Idempotent. NOT called on deactivate (a node's handle survives + * activate↔deactivate and is reused on reactivation; only a rewire swaps it). + */ + unregister(handle: Handle): void { + this.pools[handle.poolId].unregister(handle.handleId); + // Drop the stat UNCONDITIONALLY (QA F2.1), not only while recording: if recording was + // OFF at unregister time the key would linger, and a later register reusing this id (the + // free-list) would inherit the prior tenant's counters once recording resumes. delete of + // an absent key is a cheap no-op; unregister is off the hot invoke path (F-PERF intact). + this._stats.delete(statKey(handle)); + } + /** Uniform sync-void invoke (R-sync-core / R-dispatch-all). */ invoke(handle: Handle, ctx: Ctx): void { if (!this._recording) { diff --git a/packages/ts/src/graph/graph.ts b/packages/ts/src/graph/graph.ts index 445bccad..d754f9ca 100644 --- a/packages/ts/src/graph/graph.ts +++ b/packages/ts/src/graph/graph.ts @@ -173,23 +173,16 @@ export class Graph { ): Node { // effect cleanup = deactivation-only (D28 / Flag 3): the LATEST returned cleanup // fires ONCE when the effect deactivates (not between re-runs — onRerun was cut). - // State lives in ctx.state (per-node, wiped on fresh-lifecycle) so the deactivation - // hook is registered exactly once per activation and re-registers after a re-activation. - interface EffectState { - registered: boolean; - cleanup?: () => void; - } + // R-cleanup-hooks per-run lifecycle (D28 clarification / C-14): the substrate clears + // the hook list before EACH fn run, so the effect re-registers its returned cleanup + // every run — only the latest run's registration is live (or none, if the latest run + // returned void). No st.registered/ctx.state bookkeeping; re-registering every run is + // safe (the per-run clear prevents accumulation). const ctxFn: NodeFn = (ctx: Ctx) => { try { const args = ctx.depRecords.map((r) => r.latest) as DepValues; const cleanup = fn(...args); - const st: EffectState = ctx.state.get() ?? { registered: false }; - st.cleanup = typeof cleanup === "function" ? cleanup : undefined; - if (!st.registered) { - st.registered = true; - ctx.onDeactivation(() => st.cleanup?.()); - } - ctx.state.set(st); + if (typeof cleanup === "function") ctx.onDeactivation(cleanup); } catch (e) { ctx.down([["ERROR", e]]); } @@ -228,8 +221,8 @@ export class Graph { deps: readonly Node[], opts: SugarOpts = {}, ): Node { - // Node is invariant (T appears in equals); widen the typed deps to the erased Node - // surface the free initNode / _add accept (same cast the old per-operator methods used). + // Node is invariant (T appears in NodeOptions.initial); widen the typed deps to the + // erased Node surface the free initNode / _add accept (same cast the old methods used). const erased = deps as readonly Node[]; const n = initNode(op, erased, this._nodeOpts(opts)); return this._add(n, op.factory, erased, opts); diff --git a/packages/ts/src/graph/operators.ts b/packages/ts/src/graph/operators.ts index ce79d79a..a0c90aef 100644 --- a/packages/ts/src/graph/operators.ts +++ b/packages/ts/src/graph/operators.ts @@ -47,7 +47,7 @@ export interface Operator { /** * Instantiate an operator as a bare `Node` — NO Graph required (D43 bare-node path). * Wraps `op.body` in the D30 value-throw→ERROR boundary; merges `op.opts` with caller `opts` - * (caller wins, so `partial`/`pool`/`equals`/`dispatcher` overrides take effect). The + * (caller wins, so `partial`/`pool`/`dispatcher` overrides take effect). The * dispatcher defaults to the process-global (D26) unless `opts.dispatcher` is passed. * * `g.initNode` is the same call PLUS graph registration; graph inspection diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index 673d9b08..5405edee 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -3,13 +3,14 @@ * * Holds a fn handle + deps + the wave state machine; ZERO inspection cruft * (naming/find/describe are the graph layer, CSP-2). Canonical authority: - * ~/src/graphrefly/spec/rules.jsonl (R-node-*, R-two-phase, R-diamond, R-equals, + * ~/src/graphrefly/spec/rules.jsonl (R-node-*, R-two-phase, R-diamond, R-resolved-undirty, * R-first-run-gate, R-push-subscribe, R-rom-ram, R-fn-contract, R-initial, R-ctx-up). * * Slice 1 = core wave: state node, compute node, two-phase DIRTY->DATA, diamond - * pending-counter join, first-run gate, equals->RESOLVED, push-on-subscribe, - * lazy activation, ROM/RAM. Lifecycle (terminal/INVALIDATE/cleanup), control - * (PAUSE/async), batch, and dynamicNode land in later slices. + * pending-counter join, first-run gate, substrate-synthesized undirty RESOLVED + * (R-resolved-undirty / D49 — no equals-substitution; every occurrence is DATA), + * push-on-subscribe, lazy activation, ROM/RAM. Lifecycle (terminal/INVALIDATE/cleanup), + * control (PAUSE/async), batch, and dynamicNode land in later slices. */ import { currentBatch, deferToBatch } from "../batch/batch.js"; @@ -35,8 +36,6 @@ export type Status = export interface NodeOptions { /** Pre-populate cache; source pushes [DATA, initial] on subscribe (R-initial). `null` is valid. */ initial?: T | null; - /** Custom equality for the DATA->RESOLVED substitution (R-equals). Default Object.is. */ - equals?: (a: T, b: T) => boolean; /** First-run gate off when true; fn body must guard SENTINEL per dep (R-first-run-gate). */ partial?: boolean; /** A dep terminal also settles the first-run gate (Reduce-class, R-first-run-gate). */ @@ -63,14 +62,11 @@ export interface NodeOptions { name?: string; } -const defaultEquals = Object.is as (a: unknown, b: unknown) => boolean; - export class Node { private _deps: Node[]; private _handle: Handle | null; private readonly _pool: "sync" | "async"; private readonly _dispatcher: Dispatcher; - private readonly _equals: (a: T, b: T) => boolean; private readonly _partial: boolean; private readonly _terminalAsRealInput: boolean; private readonly _completeWhenDepsComplete: boolean; @@ -103,6 +99,9 @@ export class Node { private _status: Status = "sentinel"; private _hasCalledFnOnce = false; private _emittedDirtyThisWave = false; + // R-resolved-undirty (D49): set when the fn emits any tier-3+ settle this wave; if it + // stays false after a DIRTY'd fn run, the substrate synthesizes one undirty RESOLVED. + private _emittedSettleThisWave = false; private _insideRunWave = false; /** R-rewire: reentrancy guard for setDeps/addDep/removeDep (one mutation in flight). */ private _inDepMutation = false; @@ -140,7 +139,6 @@ export class Node { ) { this._deps = deps; this._dispatcher = opts.dispatcher ?? defaultDispatcher; - this._equals = (opts.equals ?? defaultEquals) as (a: T, b: T) => boolean; this._partial = opts.partial ?? false; this._terminalAsRealInput = opts.terminalAsRealInput ?? false; this._completeWhenDepsComplete = opts.completeWhenDepsComplete ?? true; @@ -327,8 +325,14 @@ export class Node { this._rewireRunPending = false; let zeroDepUnDirty = false; try { - // fn swap (SD-1): re-register against the same pool. + // fn swap (SD-1): re-register against the same pool, then release the old handle + // (B15) so the rewired-away fn closure is GC'd and its dispatcher slot is reused — + // a rewire-heavy graph (CSP-2.7 *Map) no longer leaks a handle per swap. Register + // first, then unregister the old: this._handle never points at a freed slot, and a + // null old handle (a passthrough/state node gaining a fn) has nothing to free. + const oldHandle = this._handle; this._handle = this._dispatcher.register(fn, this._pool); + if (oldHandle !== null) this._dispatcher.unregister(oldHandle); const removed = oldDeps.filter((d) => !newDeps.includes(d)); let removedDirtyContributor = false; @@ -400,9 +404,10 @@ export class Node { } // Q6 auto-settle: removing the sole dirty contributor closes the wave. With deps - // remaining, request the atomic settle (recompute; equals absorbs a no-change run → - // RESOLVED). With zero deps the node is inert (degenerate fn-no-deps) — just un-dirty - // downstream. Cache is preserved either way (Q7). + // remaining, request the atomic settle (recompute → DATA for a value; a no-emit fn + // gets a substrate-synthesized undirty RESOLVED per R-resolved-undirty/D49 — NOT + // equals-absorption, which is gone). With zero deps the node is inert (degenerate + // fn-no-deps) — just un-dirty downstream. Cache is preserved either way (Q7). if (removedDirtyContributor && this._pending === 0 && this._status === "dirty") { if (newDeps.length > 0) this._rewireRunPending = true; else zeroDepUnDirty = true; @@ -515,6 +520,15 @@ export class Node { this._depDirty[idx] = false; this._pending--; } + // D50 / R-paused-invalidate: this INVALIDATE SUPERSEDES the dep's buffered + // paused dep-wave (_depBatch[idx] just cleared). Re-derive the paused-recompute + // flag — if no dep still carries a buffered DATA, CANCEL the paused recompute + // (attributed cancellation; the node has settled to SENTINEL via its own + // INVALIDATE, so a RESUME must not recompute against a now-SENTINEL dep). A + // surviving dep keeps it set; a later DATA re-arms it ([DATA,INVALIDATE,DATA2]). + if (this._pausedDepWaveOccurred && this._depBatch.every((b) => b === null)) { + this._pausedDepWaveOccurred = false; + } const hadData = this._hasData; this._invalidate(); // cascades INVALIDATE iff populated; no-op otherwise // If we broadcast DIRTY this wave but _invalidate produced no settle (the node @@ -671,6 +685,14 @@ export class Node { const b = this._depBatch[0]; if (b !== null && b.length > 0) { this._down([["DATA", b[b.length - 1]]]); + } else if (this._emittedDirtyThisWave) { + // R-resolved-undirty (D49): the dep settled via an undirty RESOLVED (no DATA in the + // batch), but this wire already broadcast DIRTY downstream this wave — balance it with + // a RESOLVED so downstream un-dirties instead of wedging. Routed through _down (NOT a + // bare _emitToSubs) so the balance respects batch-defer (D12) + pause-buffer, matching + // the zero-dep un-dirty path. Without this, a passthrough over a filter-reject / + // distinctUntilChanged-dup leaves a dangling DIRTY (the wedge D49 made common). + this._down([["RESOLVED"]]); } this._depBatch[0] = null; this._emittedDirtyThisWave = false; @@ -688,7 +710,16 @@ export class Node { "synchronous feedback cycle: node fn re-entered its own wave (R-reentrancy / D37)", ); this._hasCalledFnOnce = true; + // R-cleanup-hooks per-run lifecycle (D28 clarification): clear BOTH hook lists + // before the fn runs; the fn body re-registers the current run's hooks. Only the + // latest run's registrations are live — a re-run supersedes the prior run's hooks, + // discarded WITHOUT firing (no fire-on-rerun; onRerun stays cut). Fixes the push-only + // accumulation (K stale hooks fired after K runs). C-14. + this._onInvalidate = []; + this._onDeactivation = []; const ctx = this._buildCtx(); + const wasDirty = this._emittedDirtyThisWave; + this._emittedSettleThisWave = false; this._insideRunWave = true; try { this._dispatcher.invoke(this._handle as Handle, ctx); @@ -696,6 +727,25 @@ export class Node { this._insideRunWave = false; } + // R-resolved-undirty (D49): a SYNC fn DIRTY'd in phase 1 that produced NO tier-3 value + // this wave (filter-reject / distinctUntilChanged-dup / any no-emit fn) gets a substrate- + // SYNTHESIZED undirty RESOLVED to clear the downstream dirty — operator bodies stay + // protocol-clean (R-primary-api-clean). Status reflects cache freshness: a carried value + // -> resolved, never-valued -> sentinel. EXEMPT: terminal/INVALIDATE waves (they set + // _emittedSettleThisWave and balance their own dirty), and ASYNC-pool nodes — an async fn + // that returns without emitting has DEFERRED its result (it emits later via the stashed + // ctx), NOT rejected; synthesizing here would prematurely settle a still-pending diamond + // leg (R-async-paused / C-4). The eventual async ctx.down carries its own DIRTY balance. + if ( + wasDirty && + !this._emittedSettleThisWave && + this._terminal === undefined && + !this._isAsyncPool() + ) { + this._status = this._hasData ? "resolved" : "sentinel"; + this._emitToSubs(["RESOLVED"]); + } + // roll wave-local state forward for (let i = 0; i < this._depBatch.length; i++) this._depBatch[i] = null; this._emittedDirtyThisWave = false; @@ -738,8 +788,9 @@ export class Node { }, }; if (this._dynamic) { - // R-dynamic-node: read a dep's latest by index. Untracked deps still drive - // waves; if the output is unchanged, equals absorbs them (RESOLVED). + // R-dynamic-node: read a dep's latest by index. Untracked deps still drive waves and + // re-run the fn; under D49 (no equals-substitution) the fn re-emits its current value + // as DATA — to suppress redundant downstream propagation, pair with distinctUntilChanged. ctx.track = (i: number) => ctx.depRecords[i]?.latest; } return ctx; @@ -760,6 +811,19 @@ export class Node { // ── downstream emission pipeline (the unified waist) ── private _down(msgs: Wave): void { + // Terminal-is-forever (R-terminal / D17 / B30): once COMPLETE/ERROR has been emitted the + // node is final — a self-emit (state.set / ctx.down) in a LATER wave is a no-op, never + // resurrecting the cache or re-emitting. The COMPLETE/ERROR arms below also self-guard + // against a double terminal, but DATA/RESOLVED/INVALIDATE had no entry guard, so a + // post-terminal set() would overwrite cache + emit DATA. The upstream path + // (_receiveFromDep) already drops on terminal the same way — including a TEARDOWN that + // follows the node's own COMPLETE: whether a terminal intermediate should still relay + // TEARDOWN for downstream unwire is an OPEN spec gap on draft R-teardown-complete (a + // /spec-amend call, not a guard tweak; today both the TS and Rust arms drop it). This + // closes the self-emit gap and matches the Rust arm's Core::down blanket guard. A single + // wave that goes terminal mid-loop (e.g. [COMPLETE, TEARDOWN]) is unaffected: _terminal + // is still undefined at entry. Resubscribable reset clears _terminal before any re-emit. + if (this._terminal !== undefined) return; let sorted: Message[] = [...msgs].sort((a, b) => messageTier(a[0]) - messageTier(b[0])); // R-same-wave-merge: collapse repeated INVALIDATE in one wave (Q9) so the // cleanup hook + downstream broadcast fire at most once. @@ -814,10 +878,12 @@ export class Node { if (m[0] === "RESOLVED") hasResolved = true; if (messageTier(m[0]) === 3) hasTier3 = true; } - // EC2 / R-equals tier-3 exclusivity: a wave's tier-3 slot is >=1 DATA XOR exactly - // 1 RESOLVED — never mixed. Reject the protocol violation fail-fast. + // EC2 / R-resolved-undirty tier-3 exclusivity: a wave's tier-3 slot is >=1 DATA + // (occurrence) XOR exactly 1 RESOLVED (undirty) — never mixed. Reject fail-fast. if (dataCount >= 1 && hasResolved) { - throw new Error("down: a wave cannot mix DATA and RESOLVED (tier-3 exclusivity, R-equals)"); + throw new Error( + "down: a wave cannot mix DATA and RESOLVED (tier-3 exclusivity, R-resolved-undirty)", + ); } // Synthesize a leading DIRTY for an EXTERNAL tier-3 emit (R-dirty-before-data). @@ -829,6 +895,9 @@ export class Node { } for (const m of sorted) { + // R-resolved-undirty (D49): a tier-3+ emit this wave means the fn produced a settle, + // so no synthesized undirty RESOLVED is owed (see _runWave). + if (messageTier(m[0]) >= 3) this._emittedSettleThisWave = true; if (m[0] === "DIRTY") { if (!this._emittedDirtyThisWave) { this._emittedDirtyThisWave = true; @@ -843,20 +912,17 @@ export class Node { throw new Error("down: DATA requires a non-SENTINEL payload (R-data-payload)"); } const v = m[1] as T; - // R-equals: DATA->RESOLVED substitution ONLY on a single-DATA wave. - if (dataCount === 1 && this._hasData && this._equals(this._cache as T, v)) { - this._status = "resolved"; - this._emitToSubs(["RESOLVED"]); - } else { - this._cache = v; - this._hasData = true; - this._status = "settled"; - if (this._replayN > 0) { - this._replayRing.push(v); - if (this._replayRing.length > this._replayN) this._replayRing.shift(); - } - this._emitToSubs(["DATA", v]); + // R-resolved-undirty (D49): every value-occurrence is emitted as DATA — the + // substrate never substitutes DATA->RESOLVED on value-equality. Dedup is opt-in + // at the operator layer (distinctUntilChanged), never a substrate behavior. + this._cache = v; + this._hasData = true; + this._status = "settled"; + if (this._replayN > 0) { + this._replayRing.push(v); + if (this._replayRing.length > this._replayN) this._replayRing.shift(); } + this._emitToSubs(["DATA", v]); continue; } if (m[0] === "RESOLVED") { @@ -1073,8 +1139,9 @@ export function node( /** * Construct a dynamicNode (R-dynamic-node / D35) — a node variant whose fn reads a * subset of a fixed superset of deps per invocation via `ctx.track(i)`. All declared - * deps participate in wave tracking; an unread dep's change leaves the output unchanged, - * so equals absorbs it (no downstream propagation). Intra-graph only (D22). + * deps participate in wave tracking; an unread dep's change re-runs the fn, which re-emits + * its current value as DATA (D49 removed equals-absorption — dedup is opt-in via + * distinctUntilChanged). Intra-graph only (D22). */ export function dynamicNode( deps: Node[], From f79a6f0415e32c2ff3fa2aaa61faed120b2d6b9f Mon Sep 17 00:00:00 2001 From: David Chen Date: Sat, 30 May 2026 12:36:55 -0700 Subject: [PATCH 010/175] feat(ts): implement higher-order operators and rewire functionality - Added higher-order operators: `switchMap`, `mergeMap`, `concatMap`, `exhaustMap`, and `flatMap` to enhance data handling capabilities. - Introduced `ctx.rewireNext` for deferred self-rewiring, allowing nodes to modify their dependency sets at the committed wave boundary, preventing mid-run topology changes. - Updated `index.ts` to export new types and operators, improving type management and usability. - Created comprehensive tests for the new operators and rewire functionality, ensuring correct behavior and compliance with the deferred rewire mechanism. - Enhanced the `Node` class to support the new higher-order operations and maintain proper state during rewiring processes. --- packages/ts/src/__tests__/conformance.test.ts | 310 +++++++++++++++++- .../ts/src/__tests__/higher-order.test.ts | 288 ++++++++++++++++ .../ts/src/__tests__/rewire-deferred.test.ts | 286 ++++++++++++++++ packages/ts/src/batch/batch.ts | 57 ++-- packages/ts/src/batch/boundary.ts | 86 +++++ packages/ts/src/ctx/types.ts | 26 ++ packages/ts/src/graph/graph.ts | 47 ++- packages/ts/src/graph/higher-order.ts | 187 +++++++++++ packages/ts/src/graph/operators.ts | 6 +- packages/ts/src/index.ts | 10 +- packages/ts/src/node/node.ts | 223 +++++++++++-- 11 files changed, 1462 insertions(+), 64 deletions(-) create mode 100644 packages/ts/src/__tests__/higher-order.test.ts create mode 100644 packages/ts/src/__tests__/rewire-deferred.test.ts create mode 100644 packages/ts/src/batch/boundary.ts create mode 100644 packages/ts/src/graph/higher-order.ts diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index 5e726dc3..0fb8b513 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -9,8 +9,8 @@ */ import { describe, expect, it } from "vitest"; -import type { Ctx, Message } from "../index.js"; -import { distinctUntilChanged, filter, fromIter, graph, node, take } from "../index.js"; +import type { Ctx, Message, NodeFn } from "../index.js"; +import { distinctUntilChanged, filter, fromIter, graph, type Node, node, take } from "../index.js"; const types = (msgs: Message[]) => msgs.map((m) => m[0]); const data = (msgs: Message[]) => @@ -325,6 +325,204 @@ describe("C-10 true-mode async leaf source delivers immediately under PAUSE (R-p }); }); +// C-11 (D47 / R-rewire-deferred): a node fn's SELF-triggered dep-set mutation via ctx.rewireNext +// is deferred to the committed wave boundary (never mutating _deps mid-run, never the D37 in-fn +// reject), drained as a fresh wave; added cached inners push [DIRTY,DATA] without re-arming the +// gate; removed inners are drained + _deactivate (onDeactivation = abortInFlight). The substrate +// prerequisite for the higher-order *Map operators. Distinct from C-8 (external/immediate rewire). +describe("C-11 higher-order inner rewire at the wave boundary (R-rewire-deferred / D47)", () => { + // Inners are leaf sources whose activation + deactivation are observable (cancellation visible). + function makeInner(seed?: number) { + let ictx: Ctx | null = null; + let activated = false; + let deactivated = false; + const n = node([], (ctx) => { + ictx = ctx; + activated = true; + ctx.onDeactivation(() => { + deactivated = true; + }); + if (seed !== undefined) ctx.down([["DATA", seed]]); + }); + return { + node: n, + emit: (v: number) => (ictx as Ctx).down([["DATA", v]]), + complete: () => (ictx as Ctx).down([["COMPLETE"]]), + isActivated: () => activated, + isDeactivated: () => deactivated, + }; + } + + // A merge-style OP: spawn+add an inner per S DATA, forward inner DATA, remove a completed inner. + function mergeOp(s: Node) { + const inners: Node[] = []; + const opFn: NodeFn = (ctx) => { + const removals: Node[] = []; + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + if (ctx.depRecords[i].terminal === true) removals.push(inners[i - 1]); + } + const sv = ctx.depRecords[0].batch; + if (sv && sv.length > 0) { + const inner = makeInner((sv[sv.length - 1] as number) * 10); + inners.push(inner.node); + ctx.rewireNext.addDep(inner.node, opFn); + } + for (const r of removals) { + inners.splice(inners.indexOf(r), 1); + ctx.rewireNext.removeDep(r, opFn); + } + }; + return node([s], opFn, { + completeWhenDepsComplete: false, + terminalAsRealInput: true, + }); + } + + it("(steps 1-3) addDep deferred to the boundary; added cached inner pushes [DIRTY,DATA], gate not re-armed", () => { + const s = node([], null); + let opRuns = 0; + const inners: Node[] = []; + const opFn: NodeFn = (ctx) => { + opRuns++; + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + } + const sv = ctx.depRecords[0].batch; + if (sv && sv.length > 0) { + const inner = makeInner((sv[sv.length - 1] as number) * 10); + inners.push(inner.node); + // mid-run: _deps is NOT mutated — the inner is not yet wired/activated. + expect(inner.isActivated()).toBe(false); + ctx.rewireNext.addDep(inner.node, opFn); + expect(inner.isActivated()).toBe(false); // still deferred after the request + } + }; + const op = node([s], opFn, { + completeWhenDepsComplete: false, + terminalAsRealInput: true, + }); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // step 1: request addDep(innerA); step 2: drain wires it; step 3: forward + expect(types(msgs)).toContain("DIRTY"); // step 2 boundary wave is two-phase… + expect(data(msgs)).toEqual([10]); // …innerA's seed (1*10) forwarded as DATA + + opRuns = 0; + s.down([["DATA", 2]]); // gate NOT re-armed: S alone re-drives the fn (and adds innerB) + expect(opRuns).toBeGreaterThan(0); + }); + + it("(step 7) an inner COMPLETE removes it (bounding); OP does not COMPLETE while S is live", () => { + const s = node([], null); + const op = mergeOp(s); + const { msgs } = collect(op); + + s.down([["DATA", 5]]); // add innerA (seed 50 forwarded) + expect(data(msgs)).toContain(50); + msgs.length = 0; + + s.down([["DATA", 6]]); // add innerB (seed 60 forwarded) + expect(data(msgs)).toContain(60); + + // OP stays live: S is still live, completeWhenDepsComplete:false → no terminal cascade. + expect(op.status).not.toBe("completed"); + expect(types(msgs)).not.toContain("COMPLETE"); + }); + + it("(steps 4-6, switch) setDeps tears down the superseded inner's source and forwards only the new one", () => { + const s = node([], null); + const innerA = makeInner(10); + const innerB = makeInner(20); + let current: Node | null = null; + const opFn: NodeFn = (ctx) => { + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + } + const sv = ctx.depRecords[0].batch; + if (sv && sv.length > 0) { + current = (sv[sv.length - 1] as number) === 1 ? innerA.node : innerB.node; + ctx.rewireNext.setDeps([s, current], opFn); + } + }; + const op = node([s], opFn, { + completeWhenDepsComplete: false, + terminalAsRealInput: true, + }); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // → innerA live (seed 10 forwarded) + expect(data(msgs)).toContain(10); + expect(innerA.isActivated()).toBe(true); + msgs.length = 0; + + s.down([["DATA", 2]]); // switch → innerB; innerA's SOURCE torn down (not masked) + expect(innerB.isActivated()).toBe(true); + expect(innerA.isDeactivated()).toBe(true); + expect(data(msgs)).toEqual([20]); // ONLY the current inner forwarded + msgs.length = 0; + + innerA.emit(999); // the superseded inner is DRAINED — no stale forward survives + expect(data(msgs)).toEqual([]); + }); + + it("(variant) an IMMEDIATE in-fn self-rewire is the D37 feedback cycle → graph-layer ERROR (not rewireNext)", () => { + const g = graph(); + const a = g.state(1); + const x = g.state(9); + let op: Node; + // g.derived carries the D30 value-throw→ERROR boundary; the fn does an IMMEDIATE self-addDep + // (NOT ctx.rewireNext) mid-run → the D37 reject throws → graph layer converts it to ERROR. + op = g.derived([a], (av) => { + op.addDep(x, (c) => c.down([["DATA", c.depRecords[0].latest as number]])); + return av as number; + }); + let escaped = false; + try { + op.subscribe(() => {}); + } catch { + escaped = true; // the substrate throw must be caught by the graph layer (D30), not escape + } + expect(escaped).toBe(false); + expect(op.status).toBe("errored"); + }); + + it("(variant) a terminal OP discards its pending rewireNext queue", () => { + const s = node([], null); + const inner = makeInner(1); + const opFn: NodeFn = (ctx) => { + if (ctx.depRecords[0].batch) { + ctx.rewireNext.addDep(inner.node, opFn); // queued… + ctx.down([["COMPLETE"]]); // …then OP goes terminal THIS wave + } + }; + const op = node([s], opFn, { + completeWhenDepsComplete: false, + terminalAsRealInput: true, + }); + collect(op); + s.down([["DATA", 1]]); + expect(op.status).toBe("completed"); + expect(inner.isActivated()).toBe(false); // queued addDep discarded + }); + + it("(variant) a no-net-change rewireNext is a no-op (no drain loop)", () => { + const a = node([], null, { initial: 1 }); + let runs = 0; + const op = node([a], function opFn(ctx) { + runs++; + if (runs < 5) ctx.rewireNext.setDeps([a], opFn); // identical dep set every run + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + collect(op); + expect(runs).toBe(1); // the idempotent setDeps changes nothing → no fresh wave → no loop + expect(op.cache).toBe(1); + }); +}); + // C-12 (D49 / R-resolved-undirty, supersedes D15/R-equals): every value-occurrence is DATA // (no auto-equals-substitution); RESOLVED is the substrate-SYNTHESIZED undirty-only signal; // dedup is opt-in at the operator layer. Folds the former _probe.test.ts probes (the @@ -529,3 +727,111 @@ describe("C-14 cleanup hooks are per-run (cleared + re-registered each fn run) ( expect(cleanup).toBe(1); }); }); + +// C-15 (R-terminal-settles-dirty / B35): a dep's terminal (COMPLETE/ERROR) releases its +// outstanding in-wave DIRTY contribution — the exactly-one-settle invariant — exactly as +// DATA/RESOLVED/INVALIDATE do, so a DIRTY-then-terminal-without-DATA dep never strands the +// join node's pending and wedges it. The terminal analogue of the INVALIDATE wedge-guard. +describe("C-15 a dep's terminal releases its in-wave DIRTY contribution (R-terminal-settles-dirty / D-none)", () => { + const sum2 = (ctx: Ctx) => + ctx.down([ + [ + "DATA", + ((ctx.depRecords[0].latest as number) ?? 0) + ((ctx.depRecords[1].latest as number) ?? 0), + ], + ]); + + it("(a) COMPLETE-mid-dirty: D joins ONCE on the live leg, not wedged", () => { + const b = node([], null, { initial: 1 }); + const c = node([], null, { initial: 10 }); + const d = node([b, c], sum2, { completeWhenDepsComplete: false }); + const { msgs } = collect(d); + expect(d.cache).toBe(11); // first join (activation): 1 + 10 + msgs.length = 0; + + b.down([["DIRTY"]]); // B signals a change toward D (phase 1) → D dirty, pending=1 + expect(d.status).toBe("dirty"); + c.down([["DATA", 20]]); // C delivers a real value — but D is gated on B's pending + expect(d.cache).toBe(11); // not yet (B still dirty) + b.down([["COMPLETE"]]); // B COMPLETEs with NO DATA → releases B's dirty → D joins on C + + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // joined EXACTLY once, glitch-free, no wedge + expect(d.cache).toBe(21); // B's last value (1) + C's new value (20) + expect(d.status).not.toBe("completed"); // C is still live (completeWhenDepsComplete:false) + }); + + it("(b) sole-dirty: B's COMPLETE drains pending → D un-dirties via RESOLVED (no fabricated DATA)", () => { + const b = node([], null, { initial: 1 }); + const c = node([], null, { initial: 10 }); + const d = node([b, c], sum2, { completeWhenDepsComplete: false }); + const { msgs } = collect(d); + expect(d.cache).toBe(11); + msgs.length = 0; + + b.down([["DIRTY"]]); // B the SOLE dirty contributor (C unchanged this wave) + b.down([["COMPLETE"]]); // COMPLETEs with no value → no occurrence → undirty RESOLVED + + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); // un-dirtied, NOT a fabricated DATA + expect(d.cache).toBe(11); // cache preserved (a terminal, unlike INVALIDATE, keeps the value) + expect(d.status).toBe("resolved"); // undirty-RESOLVED convention (hasData → resolved), not "settled" + }); + + it("(c) rescue: an absorbed ERROR releases the dirty + the fn reads the terminal, no wedge", () => { + const b = node([], null, { initial: 1 }); + const c = node([], null, { initial: 10 }); + const rescue = (ctx: Ctx) => { + const bt = ctx.depRecords[0].terminal; + const bv = bt !== undefined && bt !== true ? 0 : ((ctx.depRecords[0].latest as number) ?? 0); + ctx.down([["DATA", bv + ((ctx.depRecords[1].latest as number) ?? 0)]]); + }; + const d = node([b, c], rescue, { + errorWhenDepsError: false, + terminalAsRealInput: true, + }); + const { msgs } = collect(d); + expect(d.cache).toBe(11); + msgs.length = 0; + + b.down([["DIRTY"]]); // B dirties D + b.down([["ERROR", new Error("boom")]]); // rescued (errorWhenDepsError:false) → fn reads terminal + + expect(d.status).not.toBe("errored"); // NOT propagated — rescued + expect(data(msgs)).toEqual([10]); // B rescued to 0 + C(10); released dirty, no stranded pending + }); + + it("(d) gate-holds: a dirtied dep completing-empty on a PRE-first-run multi-dep node un-dirties, never wedges", () => { + // QA gate-holds corner (mutation-verified): node not yet first-run; C dirties then COMPLETEs + // with NO value while B has DATA — but the first-run gate STILL holds (C never delivered, + // terminalAsRealInput:false). The fn cannot run, yet the broadcast DIRTY must be balanced by + // a RESOLVED, else downstream wedges (the B35 class missed by the naive sawData→_maybeRun). + let runs = 0; + const b = node([], null); // no initial — delivers only when driven + const c = node([], null); + const d = node( + [b, c], + (ctx: Ctx) => { + runs++; + ctx.down([ + [ + "DATA", + ((ctx.depRecords[0].latest as number) ?? 0) + + ((ctx.depRecords[1].latest as number) ?? 0), + ], + ]); + }, + { completeWhenDepsComplete: false }, + ); + const { msgs } = collect(d); + expect(runs).toBe(0); // gate holds — neither dep delivered yet + msgs.length = 0; + + c.down([["DIRTY"]]); // C signals a change → D dirty, broadcasts DIRTY (pending=1) + b.down([["DATA", 5]]); // B delivers a value, but the gate still needs C → D gated + expect(runs).toBe(0); + c.down([["COMPLETE"]]); // C COMPLETEs with NO value → releases dirty; gate STILL holds + + expect(runs).toBe(0); // fn never ran (C never delivered data, terminalAsRealInput:false) + expect(types(msgs)).toEqual(["DIRTY", "RESOLVED"]); // un-dirtied downstream, NOT wedged + expect(d.cache).toBeUndefined(); // never produced a value + }); +}); diff --git a/packages/ts/src/__tests__/higher-order.test.ts b/packages/ts/src/__tests__/higher-order.test.ts new file mode 100644 index 00000000..a6651c66 --- /dev/null +++ b/packages/ts/src/__tests__/higher-order.test.ts @@ -0,0 +1,288 @@ +/** + * Higher-order operators — switchMap / mergeMap / concatMap / exhaustMap / flatMap + * (D47 / R-rewire-deferred / CSP-2.7). Per-language sugar (D6/D24); built on the deferred + * self-rewire substrate (ctx.rewireNext) + the g.initNode funnel (D43). Each test drives a + * controllable inner "subject" so the flatten policy + inner-lifecycle folding are observable. + */ + +import { describe, expect, it } from "vitest"; +import type { Ctx, Message } from "../index.js"; +import { + concatMap, + exhaustMap, + flatMap, + graph, + initNode, + mergeMap, + node, + of, + switchMap, +} from "../index.js"; + +function collect(n: { subscribe(s: (m: Message) => void): () => void }) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const data = (m: Message[]) => + m.filter((x) => x[0] === "DATA").map((x) => (x as ["DATA", unknown])[1]); + +/** A controllable inner source whose activation + deactivation (cancellation) are observable. */ +function subject() { + let ctxRef: Ctx | null = null; + let activated = false; + let deactivated = false; + const n = node([], (ctx) => { + ctxRef = ctx; + activated = true; + ctx.onDeactivation(() => { + deactivated = true; + }); + }); + return { + node: n, + next: (v: number) => (ctxRef as Ctx).down([["DATA", v]]), + complete: () => (ctxRef as Ctx).down([["COMPLETE"]]), + error: (e: unknown) => (ctxRef as Ctx).down([["ERROR", e]]), + isActivated: () => activated, + isDeactivated: () => deactivated, + }; +} + +describe("mergeMap (D47 / R-rewire-deferred)", () => { + it("keeps all inners live and interleaves their emissions; a completed inner is bounded", () => { + const a = subject(); + const b = subject(); + const g = graph(); + const s = g.node([], null); // manual source + const op = g.initNode( + mergeMap((v: number) => (v === 1 ? a.node : b.node)), + [s], + ); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // → inner a added + s.down([["DATA", 2]]); // → inner b added (a stays live — merge) + expect(a.isActivated()).toBe(true); + expect(b.isActivated()).toBe(true); + + a.next(10); + b.next(20); + a.next(11); + expect(data(msgs)).toEqual([10, 20, 11]); // interleaved (concurrent inners) + + a.complete(); // inner a done → removeDep → deactivated (bounding); b unaffected + expect(a.isDeactivated()).toBe(true); + expect(b.isDeactivated()).toBe(false); + + b.next(21); + expect(data(msgs)).toEqual([10, 20, 11, 21]); + }); + + it("a projector returning an ALREADY-LIVE inner is merged once (no inners↔deps desync)", () => { + // QA fix (mutation-verified): addDep is set-idempotent, so a duplicate-projected Node must + // not be double-tracked in `inners` — otherwise the inners[i]↔deps[i+1] map skews by one and + // a LATER inner COMPLETE removes the WRONG (already-gone) inner, leaking the real one. The + // leak only surfaces on a subsequent removal, so we drive X (twice → dup), then a distinct Y, + // then complete both and assert Y is actually torn down. + const x = subject(); + const y = subject(); + const g = graph(); + const s = g.node([], null); + const op = g.initNode( + mergeMap((v: number) => (v === 3 ? y.node : x.node)), + [s], + ); + collect(op); + + s.down([["DATA", 1]]); // add x + s.down([["DATA", 2]]); // re-project x → already live → skipped (no double-track) + s.down([["DATA", 3]]); // add a DISTINCT inner y + x.complete(); // remove x → deps=[S,y] + y.complete(); // remove y — only correct if inners stayed aligned (else x's stale slot is hit) + expect(x.isDeactivated()).toBe(true); + expect(y.isDeactivated()).toBe(true); // WITHOUT the dedup guard this stays false (wrong slot removed) + }); + + it("flatMap is an alias of mergeMap", () => { + const a = subject(); + const g = graph(); + const s = g.node([], null); + const op = g.initNode( + flatMap((_v: number) => a.node), + [s], + ); + const { msgs } = collect(op); + s.down([["DATA", 1]]); + a.next(7); + expect(data(msgs)).toEqual([7]); + }); + + it("COMPLETEs when the source is done AND every inner has completed", () => { + const a = subject(); + const g = graph(); + const s = g.node([], null); + const op = g.initNode( + mergeMap((_v: number) => a.node), + [s], + ); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // add inner a + s.down([["COMPLETE"]]); // source done, but inner a still live → NOT complete yet + expect(op.status).not.toBe("completed"); + + a.next(9); + a.complete(); // last inner done + source done → op COMPLETEs + expect(data(msgs)).toEqual([9]); + expect(op.status).toBe("completed"); + }); + + it("a throwing projector → ERROR (D30, self-catch survives rewire)", () => { + const g = graph(); + const s = g.node([], null); + const op = g.initNode( + mergeMap((_v: number) => { + throw new Error("boom"); + }), + [s], + ); + collect(op); + s.down([["DATA", 1]]); + expect(op.status).toBe("errored"); + }); +}); + +describe("switchMap (D47)", () => { + it("cancels the in-flight inner on a new source value (abortInFlight) and forwards only the current", () => { + const a = subject(); + const b = subject(); + const g = graph(); + const s = g.node([], null); + const op = g.initNode( + switchMap((v: number) => (v === 1 ? a.node : b.node)), + [s], + ); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // → inner a + a.next(10); + expect(data(msgs)).toEqual([10]); + + s.down([["DATA", 2]]); // switch → inner b; a is CANCELLED (source torn down) + expect(a.isDeactivated()).toBe(true); + expect(b.isActivated()).toBe(true); + + a.next(99); // the superseded inner is drained — no stale forward + b.next(20); + expect(data(msgs)).toEqual([10, 20]); + }); +}); + +describe("concatMap (D47)", () => { + it("runs inners one at a time in source order; later source values queue", () => { + const a = subject(); + const b = subject(); + const g = graph(); + const s = g.node([], null); + const op = g.initNode( + concatMap((v: number) => (v === 1 ? a.node : b.node)), + [s], + ); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // activate inner a + s.down([["DATA", 2]]); // QUEUED — a is still active (b not yet activated) + expect(a.isActivated()).toBe(true); + expect(b.isActivated()).toBe(false); + + a.next(10); + a.complete(); // a done → activate the queued b + expect(b.isActivated()).toBe(true); + b.next(20); + expect(data(msgs)).toEqual([10, 20]); // order preserved + }); +}); + +describe("exhaustMap (D47)", () => { + it("drops source values that arrive while an inner is active", () => { + const a = subject(); + const b = subject(); + const c = subject(); + const g = graph(); + const s = g.node([], null); + const project = (v: number) => (v === 1 ? a.node : v === 2 ? b.node : c.node); + const op = g.initNode(exhaustMap(project), [s]); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // activate inner a + s.down([["DATA", 2]]); // DROPPED — a is active + expect(a.isActivated()).toBe(true); + expect(b.isActivated()).toBe(false); // value 2 never projected + + a.next(10); + a.complete(); // a done → exhaust free again + s.down([["DATA", 3]]); // now accepted → inner c + expect(c.isActivated()).toBe(true); + c.next(30); + expect(data(msgs)).toEqual([10, 30]); // the value-2 inner never contributed + }); +}); + +describe("higher-order — describe (D39 / D51): real factory name + LIVE inner topology", () => { + it("records the operator's real factory name; before any source value, only the construction edge", () => { + const a = subject(); + const g = graph(); + const s = g.node([], null, { name: "src" }); + g.initNode( + switchMap((_v: number) => a.node), + [s], + { name: "sw" }, + ); + const snap = g.describe(); + const sw = snap.nodes.find((dn) => dn.id === "sw"); + expect(sw?.factory).toBe("switchMap"); // D6/R-describe: REAL operator name, not "node" + expect(sw?.deps).toEqual(["src"]); // no inner wired yet — just the construction edge + expect(snap.edges).toContainEqual({ from: "src", to: "sw" }); + }); + + it("shows a LIVE runtime inner after the operator wires it (D51 live-dep snapshot, no dangling '?')", () => { + const a = subject(); + const g = graph(); + const s = g.node([], null, { name: "src" }); + const sw = g.initNode( + switchMap((_v: number) => a.node), + [s], + { name: "sw" }, + ); + collect(sw); + expect(g.describe().edges).toEqual([{ from: "src", to: "sw" }]); // before: construction only + + s.down([["DATA", 1]]); // switchMap wires inner `a` as a live dep (deferred rewire drains) + const snap = g.describe(); + // sw's live deps now include the inner; the inner is auto-discovered as a REAL snapshot node. + const innerEdge = snap.edges.find((e) => e.to === "sw" && e.from !== "src"); + expect(innerEdge).toBeDefined(); // a live edge inner→sw (truthful, not a dangling "?") + const innerNode = snap.nodes.find((dn) => dn.id === innerEdge?.from); + expect(innerNode).toBeDefined(); // the inner IS emitted as a node (D51 one-level auto-discovery) + expect(innerNode?.deps).toEqual([]); // shown as a leaf (transitive sub-deps = B38) + expect(snap.nodes.find((dn) => dn.id === "sw")?.deps).toContain("src"); // S still a live dep + }); + + it("auto-discovers an unregistered live dep (a bare initNode source) WITH its factory (D51 B2 / D43)", () => { + const g = graph(); + const inner = initNode(of(42), []); // BARE (free initNode, not g.*) → unregistered, factory "of" + const d = g.node([inner], (ctx) => ctx.down([["DATA", ctx.depRecords[0].latest as number]]), { + name: "d", + }); + collect(d); + const snap = g.describe(); + const dNode = snap.nodes.find((n) => n.id === "d"); + expect(dNode?.deps).toHaveLength(1); // d's single live dep = the bare inner + const innerId = dNode?.deps[0] as string; + const innerNode = snap.nodes.find((n) => n.id === innerId); + expect(innerNode).toBeDefined(); // emitted as a node, NOT a dangling "?" edge + expect(innerNode?.factory).toBe("of"); // named via NodeOptions.factory (D43-reserved, D51) + expect(snap.edges).toContainEqual({ from: innerId, to: "d" }); + }); +}); diff --git a/packages/ts/src/__tests__/rewire-deferred.test.ts b/packages/ts/src/__tests__/rewire-deferred.test.ts new file mode 100644 index 00000000..3ebb8ba1 --- /dev/null +++ b/packages/ts/src/__tests__/rewire-deferred.test.ts @@ -0,0 +1,286 @@ +/** + * Deferred SELF-rewire — ctx.rewireNext (R-rewire-deferred / D47 / CSP-2.7). + * + * Focused substrate units for the wave-boundary-deferred dep-set mutation that higher-order + * operators (switchMap/mergeMap/...) build on: defer-not-immediate, drain at the committed + * boundary, added cached inner pushes [DIRTY,DATA] with the gate NOT re-armed, removed inner + * drains + deactivates (onDeactivation = abortInFlight), terminal-discards-queue, no-net-change + * no-op, and the immediate in-fn path still throwing (D37). The exhaustive interleavings are the + * TLA+ model (~/src/graphrefly/formal/wave_rewire_deferred.tla); the canonical scenario is C-11. + */ + +import { describe, expect, it } from "vitest"; +import type { Ctx, Message, NodeFn } from "../index.js"; +import { batch, graph, type Node, node } from "../index.js"; + +function collect(n: Node) { + const msgs: Message[] = []; + const unsub = n.subscribe((m) => msgs.push(m)); + return { msgs, unsub }; +} +const types = (m: Message[]) => m.map((x) => x[0]); +const data = (m: Message[]) => + m.filter((x) => x[0] === "DATA").map((x) => (x as ["DATA", unknown])[1]); + +/** A leaf-source inner whose activation + deactivation are observable (cancellation visible). */ +function makeInner(seed?: number) { + let ictx: Ctx | null = null; + let activated = false; + let deactivated = false; + const n = node([], (ctx) => { + ictx = ctx; + activated = true; + ctx.onDeactivation(() => { + deactivated = true; + }); + if (seed !== undefined) ctx.down([["DATA", seed]]); + }); + return { + node: n, + emit: (v: number) => (ictx as Ctx).down([["DATA", v]]), + complete: () => (ictx as Ctx).down([["COMPLETE"]]), + isActivated: () => activated, + isDeactivated: () => deactivated, + }; +} + +describe("ctx.rewireNext — defer + drain (R-rewire-deferred / D47)", () => { + it("defers addDep to the boundary — NOT applied in place during the fn run", () => { + const inner = makeInner(99); + const s = node([], null); // no initial → op's fn first-runs on the driven DATA + let deferredCorrectly = false; + const op: Node = node( + [s], + function opFn(ctx) { + if (ctx.depRecords[0].batch) { + ctx.rewireNext.addDep(inner.node, opFn); + // still inside the fn run: the dep must NOT be wired yet (inner not activated). + deferredCorrectly = !inner.isActivated(); + } + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + collect(op); + + s.down([["DATA", 1]]); // drives op's fn → requests addDep → drains at this call's boundary + expect(deferredCorrectly).toBe(true); // mid-fn: inner was NOT yet a live dep + expect(inner.isActivated()).toBe(true); // post-boundary: drained → inner wired + activated + }); + + it("the added cached inner pushes [DIRTY,DATA] and is forwarded; the gate is NOT re-armed", () => { + const inner = makeInner(); // no seed — emits on demand + const s = node([], null); // no initial + let opRuns = 0; + const op: Node = node( + [s], + function opFn(ctx) { + opRuns++; + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + } + if (ctx.depRecords[0].batch) ctx.rewireNext.addDep(inner.node, opFn); + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + const { msgs } = collect(op); + s.down([["DATA", 1]]); // spawn + add inner (SENTINEL inner → START only, no forward yet) + msgs.length = 0; + opRuns = 0; + + inner.emit(7); // inner DATA → op forwards as a two-phase wave + expect(types(msgs)).toEqual(["DIRTY", "DATA"]); // glitch-free, R-dirty-before-data + expect(data(msgs)).toEqual([7]); + + // gate NOT re-armed: S alone re-drives op without waiting on the added inner again. + opRuns = 0; + s.down([["DATA", 2]]); // also spawns a SECOND inner (deferred); the fn still ran on S alone + expect(opRuns).toBeGreaterThan(0); + }); + + it("removeDep at the boundary drains the inner + fires onDeactivation (abortInFlight)", () => { + const inner = makeInner(5); + const s = node([], null); // no initial + const inners: Node[] = []; + const op: Node = node( + [s], + function opFn(ctx) { + const removals: Node[] = []; + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + if (ctx.depRecords[i].terminal === true) removals.push(inners[i - 1]); + } + if (ctx.depRecords[0].batch) { + inners.push(inner.node); + ctx.rewireNext.addDep(inner.node, opFn); + } + for (const r of removals) { + inners.splice(inners.indexOf(r), 1); + ctx.rewireNext.removeDep(r, opFn); + } + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + const { msgs } = collect(op); + s.down([["DATA", 1]]); // add inner (seed 5 forwarded on activation) + expect(inner.isActivated()).toBe(true); + expect(data(msgs)).toContain(5); + + inner.complete(); // inner COMPLETE → op requests removeDep → drains → inner deactivates + expect(inner.isDeactivated()).toBe(true); // input-side teardown observable + }); +}); + +describe("ctx.rewireNext — switch (setDeps) + terminal + no-net-change (D47)", () => { + it("setDeps atomically tears down the superseded inner and wires the new one", () => { + const innerA = makeInner(10); + const innerB = makeInner(20); + const s = node([], null); // no initial + let current: Node | null = null; + const op: Node = node( + [s], + function opFn(ctx) { + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + } + const sv = ctx.depRecords[0].batch; + if (sv && sv.length > 0) { + current = (sv[sv.length - 1] as number) === 1 ? innerA.node : innerB.node; + ctx.rewireNext.setDeps([s, current], opFn); // switch: one atomic op + } + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + const { msgs } = collect(op); + + s.down([["DATA", 1]]); // → inner = innerA (seed 10 forwarded) + expect(data(msgs)).toContain(10); + expect(innerA.isActivated()).toBe(true); + msgs.length = 0; + + s.down([["DATA", 2]]); // switch → innerB; innerA torn down (cancelled) + expect(innerB.isActivated()).toBe(true); + expect(innerA.isDeactivated()).toBe(true); // superseded inner's SOURCE torn down, not masked + expect(data(msgs)).toContain(20); // only the new inner forwarded + msgs.length = 0; + + innerA.emit(999); // the cancelled inner is DRAINED — no stale forward + expect(data(msgs)).toEqual([]); + }); + + it("a terminal OP discards its pending rewireNext queue", () => { + const inner = makeInner(1); + const s = node([], null, { initial: 0 }); + const op: Node = node( + [s], + function opFn(ctx) { + if (ctx.depRecords[0].batch) { + ctx.rewireNext.addDep(inner.node, opFn); // queued… + ctx.down([["COMPLETE"]]); // …then the OP goes terminal THIS wave + } + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + collect(op); + s.down([["DATA", 1]]); + expect(op.status).toBe("completed"); + expect(inner.isActivated()).toBe(false); // queued addDep discarded — inner never wired + }); + + it("a no-net-change rewireNext is a no-op (no recompute, no drain loop)", () => { + const a = node([], null, { initial: 1 }); + let runs = 0; + const op: Node = node([a], function opFn(ctx) { + runs++; + if (runs < 5) ctx.rewireNext.setDeps([a], opFn); // same dep set every run + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + collect(op); + // activation ran once; the idempotent setDeps drains but changes nothing → no fresh + // settle wave → no re-run. (A net-changing op re-issued every boundary WOULD loop — + // that is a user-level runaway, not asserted here.) + expect(runs).toBe(1); + expect(op.cache).toBe(1); + }); + + it("the immediate in-fn path still throws (D37) — rewireNext is the only legal self-rewire", () => { + const a = node([], null, { initial: 1 }); + const x = node([], null, { initial: 9 }); + const op: Node = node([a], function opFn(ctx) { + op.addDep(x, opFn); // IMMEDIATE self-rewire mid-fn → feedback cycle + ctx.down([["DATA", ctx.depRecords[0].latest as number]]); + }); + expect(() => collect(op)).toThrow(/mid-fn|feedback/); + }); +}); + +describe("ctx.rewireNext — drain robustness (QA: per-thunk isolation)", () => { + it("a throwing thunk (error-route hits a throwing sink) does NOT strand a sibling's rewire", () => { + // QA fix: _applyRewireNext catches an invalid op and routes [[ERROR,e]] down; if that ERROR + // broadcast reaches a throwing external sink, the escape must NOT abandon the rest of the + // global drain queue (else a sibling's queued rewire would silently fire at a LATER wave). + const sA = node([], null); + const sB = node([], null); + const innerB = makeInner(7); + // opA: its deferred op is invalid (self-dep) → _rewire throws → _applyRewireNext routes ERROR. + const opA: Node = node( + [sA], + function fnA(ctx) { + if (ctx.depRecords[0].batch) ctx.rewireNext.addDep(opA, fnA); // self-dep → throws at apply + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + // an external sink on opA that THROWS on ERROR → forces the error-route _down to escape apply(). + opA.subscribe((m) => { + if (m[0] === "ERROR") throw new Error("sink boom"); + }); + // opB: a perfectly valid deferred addDep, queued AFTER opA's in the same drain. + const opB: Node = node( + [sB], + function fnB(ctx) { + if (ctx.depRecords[0].batch) ctx.rewireNext.addDep(innerB.node, fnB); + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + collect(opB); + + // one batch → both fns run at commit → both thunks queue → drained together at the boundary. + expect(() => + batch(() => { + sA.down([["DATA", 1]]); // opA's thunk queued first + sB.down([["DATA", 1]]); // opB's thunk queued second + }), + ).toThrow(/sink boom/); // the first escape re-surfaces after the queue drains + expect(innerB.isActivated()).toBe(true); // opB's rewire STILL applied despite opA's throw + }); +}); + +describe("ctx.rewireNext — batch boundary (D47 / B24)", () => { + it("a rewireNext issued during batch commit drains AFTER the commit", () => { + const g = graph(); + const inner = makeInner(42); + const s = g.node([], null); // manual source, no initial (op's fn first-runs on the driven DATA) + const op: Node = g.node( + [s], + function opFn(ctx: Ctx) { + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) ctx.down([["DATA", v as number]]); + } + if (ctx.depRecords[0].batch) ctx.rewireNext.addDep(inner.node, opFn as NodeFn); + }, + { completeWhenDepsComplete: false, terminalAsRealInput: true }, + ); + const { msgs } = collect(op); + + batch(() => { + s.down([["DATA", 1]]); // deferred to commit; op's fn runs at commit → requests addDep + expect(inner.isActivated()).toBe(false); // NOT applied mid-batch (un-committed view) + }); + // after the batch boundary (post-commit): drained → inner wired + its seed forwarded + expect(inner.isActivated()).toBe(true); + expect(data(msgs)).toContain(42); + }); +}); diff --git a/packages/ts/src/batch/batch.ts b/packages/ts/src/batch/batch.ts index c8bbe7ba..7c3fb3e0 100644 --- a/packages/ts/src/batch/batch.ts +++ b/packages/ts/src/batch/batch.ts @@ -13,6 +13,7 @@ */ import type { Wave } from "../protocol/messages.js"; +import { enterWave, exitWave } from "./boundary.js"; /** A node target the batch can commit/rollback against (structural, avoids an import cycle). */ export interface BatchTarget { @@ -61,32 +62,40 @@ function rollback(b: ActiveBatch): void { /** Run `fn` as a batch (D12). DATA deferred to commit; throw/rollback discards. */ export function batch(fn: (bctx: BatchCtx) => R): R { - if (active !== null) { - // Nested batch joins the outer frame (one commit at the outermost exit). - const outer = active; - return fn({ + // Wave-owner boundary (R-rewire-deferred / D47): bracket the whole batch (fn + commit) so a + // ctx.rewireNext issued by a fn that runs during COMMIT drains AFTER the commit, never on the + // un-committed view. Inner waves (state.set, commit cascade) nest under this owner. + enterWave(); + try { + if (active !== null) { + // Nested batch joins the outer frame (one commit at the outermost exit). + const outer = active; + return fn({ + rollback: () => { + outer.rolledBack = true; + }, + }); + } + const b: ActiveBatch = { order: [], deferred: new Map(), rolledBack: false }; + active = b; + const bctx: BatchCtx = { rollback: () => { - outer.rolledBack = true; + b.rolledBack = true; }, - }); - } - const b: ActiveBatch = { order: [], deferred: new Map(), rolledBack: false }; - active = b; - const bctx: BatchCtx = { - rollback: () => { - b.rolledBack = true; - }, - }; - let result: R; - try { - result = fn(bctx); - } catch (e) { + }; + let result: R; + try { + result = fn(bctx); + } catch (e) { + active = null; + rollback(b); + throw e; + } active = null; - rollback(b); - throw e; + if (b.rolledBack) rollback(b); + else commit(b); + return result; + } finally { + exitWave(); } - active = null; - if (b.rolledBack) rollback(b); - else commit(b); - return result; } diff --git a/packages/ts/src/batch/boundary.ts b/packages/ts/src/batch/boundary.ts new file mode 100644 index 00000000..f5a7d9ba --- /dev/null +++ b/packages/ts/src/batch/boundary.ts @@ -0,0 +1,86 @@ +/** + * The wave-owner boundary + deferred self-rewire drain (R-rewire-deferred / D47). + * + * `ctx.rewireNext` defers a node's OWN dep-set mutation to the COMMITTED wave boundary. + * The substrate has no such boundary on the raw call stack, so this module establishes one: + * a module-global re-entrant DEPTH counter — the TS analogue of the Rust `WaveScope` / + * `with_wave_owner` (graphrefly-rs B25, flagged forward-load-bearing for exactly this). JS is + * single-threaded and waves are synchronous, so one cascade is on the stack at a time (same + * rationale as `batch.ts`'s module-global `active`). + * + * Every EXTERNAL wave origin brackets its synchronous cascade with {@link enterWave} / + * {@link exitWave}: `Node.subscribe`/`Node.down`/`Node.up` (activation, state.set, control), + * the async-pool `ctx.down`/`ctx.up` re-entry (which enters via the stashed ctx, NOT the public + * method), and `batch()` (so the drain fires AFTER commit). Re-entrant (nested) calls just + * inc/dec the counter; only the OUTERMOST exit (depth → 0) DRAINS the deferred-rewire queue. + * + * A single global FIFO yields the R-rewire-deferred drain order for free: issue order during a + * synchronous cascade IS causal order (a dep settles before its dependent's fn runs), so + * global-FIFO == per-node FIFO + causal-node order. Each queued mutation runs as a fresh wave + * whose own `ctx.rewireNext` calls re-enqueue and drain in the same loop (DrainExactlyOnce). + * + * Why a PROCESS-GLOBAL depth+queue is correct (not per-graph/per-dispatcher): per D22 a graph is + * a single causal/concurrency domain and cross-domain coordination is the ASYNC wire bridge + * (D32) — which never shares this synchronous call stack. So every thunk enqueued during one + * synchronous cascade belongs to one causal domain, and the outermost exit that drains it is that + * domain's committed boundary. This is the same single-threaded-sync basis as the existing + * module-global `batch.active`. (An UNSANCTIONED in-process cross-graph edge would break this — it + * is a D22 violation, not a supported topology.) + * + * Scope: the drain fires at the EndRun boundary the formal `wave_rewire_deferred.tla` models. + * The batch/pause drain-timing nuance (drain strictly after commit / final-lock RESUME, and not + * on a paused view) rides on the boundary being established by `batch()` + the public entries; + * the finer cross-axis (rewireNext issued inside an open batch / under a held pause lock) is + * backlog B24 — not modeled here. Zero behavior change when no `ctx.rewireNext` is ever called: + * the drain is one empty-queue check per outermost wave (F-PERF). + */ + +let depth = 0; +const queue: Array<() => void> = []; + +/** Enter a wave cascade (re-entrant). Pair with {@link exitWave} in a try/finally. */ +export function enterWave(): void { + depth++; +} + +/** Exit a wave cascade; the OUTERMOST exit (depth → 0) drains the deferred-rewire queue. */ +export function exitWave(): void { + depth--; + if (depth === 0 && queue.length > 0) drain(); +} + +/** + * Queue a deferred self-rewire application, drained at the committed boundary + * (R-rewire-deferred). The thunk applies one queued mutation to its owning node. + */ +export function deferRewire(apply: () => void): void { + queue.push(apply); +} + +function drain(): void { + // Each applied rewire runs a fresh wave (its own depth-bracketed cascade) which may itself + // enqueue more rewireNext requests — appended and drained by this same FIFO loop + // (DrainExactlyOnce). The depth++ around apply() prevents the fresh wave's own outermost + // exit from re-entering drain (this loop owns the draining). A net-changing op re-issued + // every boundary (oscillation) is a user-level runaway, like an infinite producer — NOT a + // substrate-detected error (D47). + // + // Per-thunk isolation: a thunk whose application throws (e.g. its _applyRewireNext error-route + // reaches a throwing external sink) must NOT abandon the rest of the queue — otherwise the + // stranded thunks would linger in this module-global queue and drain at an unrelated LATER + // wave boundary (a stale, misattributed rewire). Drain every thunk; re-surface the FIRST + // escape once the queue is empty so the error stays visible without corrupting the queue. + let escaped: { e: unknown } | null = null; + while (queue.length > 0) { + const apply = queue.shift() as () => void; + depth++; + try { + apply(); + } catch (e) { + if (escaped === null) escaped = { e }; + } finally { + depth--; + } + } + if (escaped !== null) throw escaped.e; +} diff --git a/packages/ts/src/ctx/types.ts b/packages/ts/src/ctx/types.ts index 5e86a543..f562f7cd 100644 --- a/packages/ts/src/ctx/types.ts +++ b/packages/ts/src/ctx/types.ts @@ -5,6 +5,7 @@ * R-fn-contract (D8/D27), R-ctx-state (D23/D29), R-cleanup-hooks (D28), R-ctx-up. */ +import type { Node } from "../node/node.js"; import type { Message, Wave } from "../protocol/messages.js"; /** A downstream sink callback (the only way to connect to a node's output). */ @@ -36,6 +37,25 @@ export interface CtxState { persist(on?: boolean): void; } +/** + * Deferred SELF-rewire (R-rewire-deferred / D47). A node fn may, DURING its run, request a + * mutation of its OWN dep set; the request is QUEUED and applied at the committed wave boundary + * (after the current wave settles / batch commit / final-lock RESUME) as a fresh wave, reusing + * R-rewire surgical/Option-C semantics (D42). This is the ONLY legal self-triggered rewire — an + * IMMEDIATE in-fn `node.addDep/removeDep/setDeps` is the D37 mid-fn feedback-cycle ERROR. An + * added cached dep pushes `[DIRTY,DATA]` on the boundary wave (R-push-subscribe); a removed dep + * is drained and, if it loses its last subscriber, `_deactivate`s + fires `onDeactivation` + * (input-side teardown — the basis for higher-order operator cancellation / abortInFlight). + */ +export interface RewireNext { + /** Defer adding a dep (paired with the re-supplied fn — positional fn-deps lock, SD-1). */ + addDep(dep: Node, fn: NodeFn): void; + /** Defer removing a dep (its source is torn down if it loses its last subscriber). */ + removeDep(dep: Node, fn: NodeFn): void; + /** Defer replacing the whole dep set (surgical: kept deps untouched, removed drained, added fresh-subscribe). */ + setDeps(deps: Node[], fn: NodeFn): void; +} + /** * The single argument to a node fn: `(ctx) => void` (R-fn-contract / D8). All emission * is explicit via `ctx.down`; there is no return-value framing. @@ -51,6 +71,12 @@ export interface Ctx { onDeactivation(fn: () => void): void; /** Flush on INVALIDATE (R-cleanup-hooks). */ onInvalidate(fn: () => void): void; + /** + * Deferred SELF-rewire (R-rewire-deferred / D47) — the substrate affordance higher-order + * operators (switchMap/mergeMap/concatMap/exhaustMap/flatMap) use to grow/shrink their own + * inner-source dep set in response to their own data. Always present; see {@link RewireNext}. + */ + rewireNext: RewireNext; /** * Read a dep's latest value by index (dynamicNode only, R-dynamic-node / D35). * Present only on dynamicNode fns; all declared deps still participate in wave diff --git a/packages/ts/src/graph/graph.ts b/packages/ts/src/graph/graph.ts index d754f9ca..58fe6a21 100644 --- a/packages/ts/src/graph/graph.ts +++ b/packages/ts/src/graph/graph.ts @@ -82,6 +82,11 @@ export class Graph { private readonly _mounts: Array<{ at: string; graph: Graph }> = []; private _seq = 0; private _clock = 0; // graph-local monotonic clock for observe seq (D26) + // D51: stable synthetic ids for unregistered live deps (runtime *Map inners) auto-discovered by + // describe(). WeakMap-cached so successive describes agree + the inner's id is freed when it is. + // A dedicated counter (NOT _seq) so a describe() call never perturbs registered-node id numbering. + private _synthSeq = 0; + private readonly _synthIds = new WeakMap, string>(); constructor(opts: GraphOptions = {}) { this.name = opts.name; @@ -230,27 +235,61 @@ export class Graph { // ── inspection: describe / observe / profile (D39) ── - /** Static structure snapshot (R-describe / D39). `_prefix` carries the mount path. */ + /** Live point-in-time structure snapshot (R-describe / D39 / D51). `_prefix` carries the mount path. */ describe(opts: DescribeOpts = {}, _prefix = ""): DescribeSnapshot { + // D51: edges derive from each node's CURRENT/LIVE deps (entry.node.deps, NOT the + // construction-time entry.deps), so a rewire (C-8 immediate / C-11 *Map deferred) is + // reflected and every edge is a real current subscription (D3). A live dep absent from this + // graph's index (a runtime *Map inner / bare fromAny node) is AUTO-DISCOVERED one level deep + // as a snapshot node with a synthesized stable id (B2; transitive sub-deps = backlog B38). + const discovered = new Map, string>(); // unregistered live dep → synth local id const localId = (n: Node): string => { const e = this._entries.get(n); - return e ? `${_prefix}${e.id}` : `${_prefix}?`; + if (e) return `${_prefix}${e.id}`; + // unregistered: a stable synthetic id, cached so successive describes + both call sites + // agree; the `~` prefix can't collide with a registered name or `factory#seq`. + let sid = this._synthIds.get(n); + if (sid === undefined) { + // the `~` prefix avoids the registered `name`/`factory#seq` space; guard the + // pathological case of a user node literally named `~factory#n` (bump until free). + do { + sid = `~${n.factory ?? "?"}#${this._synthSeq++}`; + } while (this._byId.has(sid)); + this._synthIds.set(n, sid); + } + discovered.set(n, sid); + return `${_prefix}${sid}`; }; const nodes: DescribeNode[] = []; const edges: DescribeEdge[] = []; + // pass 1: registered nodes, reading LIVE deps (localId records any unregistered inner). for (const entry of this._entries.values()) { const id = `${_prefix}${entry.id}`; + const liveIds = entry.node.deps.map(localId); const dnode: DescribeNode = { id, factory: entry.factory, status: entry.node.status, - deps: entry.deps.map(localId), + deps: liveIds, }; if (entry.name !== undefined) dnode.name = entry.name; if (entry.node.cache !== undefined) dnode.value = entry.node.cache; // absent = SENTINEL if (entry.meta !== undefined) dnode.meta = entry.meta; nodes.push(dnode); - for (const d of entry.deps) edges.push({ from: localId(d), to: id }); + for (const from of liveIds) edges.push({ from, to: id }); + } + // pass 2: emit the one-level auto-discovered inners as snapshot nodes (D51), shown as LEAVES + // (their own possibly-unregistered sub-deps are not traversed → no dangling edges; transitive + // = B38). factory from the inner's NodeOptions.factory (D43-reserved) else "?". + for (const [inner, sid] of discovered) { + const dnode: DescribeNode = { + id: `${_prefix}${sid}`, + factory: inner.factory ?? "?", + status: inner.status, + deps: [], + }; + if (inner.cache !== undefined) dnode.value = inner.cache; + nodes.push(dnode); } const snap: DescribeSnapshot = { nodes, edges }; if (this.name !== undefined) snap.name = this.name; diff --git a/packages/ts/src/graph/higher-order.ts b/packages/ts/src/graph/higher-order.ts new file mode 100644 index 00000000..27294e3b --- /dev/null +++ b/packages/ts/src/graph/higher-order.ts @@ -0,0 +1,187 @@ +/** + * Higher-order operators — switchMap / mergeMap / concatMap / exhaustMap / flatMap + * (D47 / R-rewire-deferred / CSP-2.7; per-language sugar, D6/D24, never in parity). + * + * Each projects every outer value to an INNER source and flattens the inners' emissions. The + * inner sources are wired as runtime DEPS via the deferred-self-rewire substrate affordance + * (`ctx.rewireNext`, D47) — NOT an internal subscribe (D45 bans that; it would create a describe + * island). Growing/shrinking the dep set is deferred to the committed wave boundary, so a fn + * never mutates its own topology mid-run (that is the D37 feedback-cycle ERROR). An inner that + * COMPLETEs is `removeDep`'d → its source `_deactivate`s + fires `onDeactivation`: that is the + * cancellation / abortInFlight basis (switchMap drops the superseded inner's WORK, not just its + * output; mergeMap bounds memory). Built on the bare `node` primitive via the {@link Operator} + * factory shape + the `g.initNode` funnel (D43), so the real factory name shows in describe. + * + * Lifecycle folding (D47): completeWhenDepsComplete:false + terminalAsRealInput:true — the + * operator owns completion (emit COMPLETE only when the SOURCE is done AND no inner is live / + * pending), and an inner/source terminal settles the gate so the fn observes it. Inner ERROR / + * source ERROR auto-forward via the substrate's errorWhenDepsError (default true) → the operator + * errors (terminal). NOTE: a terminal operator does not tear down its still-subscribed siblings + * (they are completed or will be GC'd with the operator); explicit source-error→teardown-all and + * non-default-dispatcher inner binding are first-cut limitations (backlog). + * + * Alignment: each run issues REMOVES before ADDS so the per-node FIFO drain keeps the operator's + * tracked inner list aligned with the live dep order across the intermediate boundary waves (a + * removed dep is silent — no settle — and applies before any added dep's push-on-subscribe wave). + */ + +import type { Ctx, NodeFn } from "../ctx/types.js"; +import type { Node } from "../node/node.js"; +import type { Operator } from "./operators.js"; +import { fromAny, type NodeInput } from "./sources.js"; + +/** Project an outer value to an inner source (a Node, Promise, (a)sync iterable, or scalar). */ +export type Project = (value: TIn) => NodeInput; + +type Mode = "merge" | "switch" | "concat" | "exhaust"; + +/** Per-node bookkeeping (ctx.state): the live inner deps (aligned with deps[1..]) + concat queue. */ +interface MapState { + inners: Node[]; + queue: TIn[]; // concatMap pending values (lazy projection); empty for the other modes +} + +/** + * The shared higher-order machinery. The operator depends on the source S at index 0; inner + * sources occupy indices 1.. (added/removed at runtime via ctx.rewireNext). The body is + * SELF-CATCHING (D30) and re-supplies itself on every rewire (the initNode wrap covers only the + * first run; the re-supplied fn must stay self-catching). + */ +function mapOperator( + factory: string, + project: Project, + mode: Mode, +): Operator { + const body: NodeFn = (ctx: Ctx) => { + try { + const st = (ctx.state.get>() ?? { inners: [], queue: [] }) as MapState; + let inners = st.inners; + const queue = st.queue; + + // 1. forward any inner DATA this wave (index-independent — flatten whichever inner fired). + for (let i = 1; i < ctx.depRecords.length; i++) { + const b = ctx.depRecords[i].batch; + if (b && b.length > 0) for (const v of b) ctx.down([["DATA", v]]); + } + + // 2. drop inners that terminated this/any wave (bounding); `inners` aligns with deps[1..]. + const toRemove: Node[] = []; + const survivors: Node[] = []; + for (let i = 0; i < inners.length; i++) { + if (ctx.depRecords[i + 1]?.terminal === true) toRemove.push(inners[i]); + else survivors.push(inners[i]); + } + inners = survivors; + + // 3. project the source's new value(s) per mode. + const toAdd: Node[] = []; + const make = (v: TIn): Node => + fromAny(project(v), { iter: true }) as Node; + const sb = ctx.depRecords[0].batch as readonly TIn[] | null; + if (sb && sb.length > 0) { + if (mode === "switch") { + // switch to the latest value: cancel every current inner EXCEPT a (re-projected) + // already-live one — the dep set is unique, so removing+re-adding the same Node + // would needlessly tear down + re-subscribe it. Superseded sources torn down. + const inner = make(sb[sb.length - 1]); + for (const live of inners) if (live !== inner) toRemove.push(live); + if (!inners.includes(inner)) toAdd.push(inner); + inners = [inner]; + } else if (mode === "merge") { + for (const v of sb) { + const inner = make(v); + // a projector returning an ALREADY-LIVE Node is already merged: addDep is + // set-idempotent, so double-tracking it in `inners` would desync the + // inners[i] <-> deps[i+1] map permanently. Skip the duplicate. + if (inners.includes(inner)) continue; + inners.push(inner); + toAdd.push(inner); + } + } else if (mode === "concat") { + for (const v of sb) queue.push(v); // lazy: project on activation + } else { + // exhaust: ignore the source while an inner is active. + if (inners.length === 0) { + const inner = make(sb[0]); + inners.push(inner); + toAdd.push(inner); + } + } + } + + // 3b. concat: activate the queue head when the single slot is free. + if (mode === "concat" && inners.length === 0 && toAdd.length === 0 && queue.length > 0) { + const inner = make(queue.shift() as TIn); + inners.push(inner); + toAdd.push(inner); + } + + ctx.state.set({ inners, queue }); + + // 4. issue REMOVES before ADDS (alignment): a removed dep is silent + applies before the + // added dep's push-on-subscribe settle wave, so the tracked list stays aligned. + for (const r of toRemove) ctx.rewireNext.removeDep(r, body); + for (const a of toAdd) ctx.rewireNext.addDep(a, body); + + // 5. completion: the SOURCE is done AND nothing is live or pending → COMPLETE (D47 folding; + // a queued/just-added inner keeps it open). A terminal here discards the deferred queue. + if ( + ctx.depRecords[0].terminal === true && + inners.length === 0 && + toAdd.length === 0 && + queue.length === 0 + ) { + ctx.down([["COMPLETE"]]); + } + } catch (e) { + ctx.down([["ERROR", e]]); // D30: a throwing projector → ERROR (self-catch survives rewire) + } + }; + + return { + factory, + body, + // D47 folding: the operator owns completion + observes inner/source terminals. + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + }; +} + +/** + * switchMap: project each source value to an inner; on a new source value, CANCEL the in-flight + * inner (its source `_deactivate`s — abortInFlight) and switch to the new one. Only the current + * inner's emissions are forwarded. + */ +export function switchMap(project: Project): Operator { + return mapOperator("switchMap", project, "switch"); +} + +/** + * mergeMap (a.k.a. flatMap): project each source value to an inner and keep ALL inners live, + * interleaving their emissions. A completed inner is removed (memory bounding). Inner ERROR + * forwards (the operator errors); siblings are not explicitly cancelled (first-cut limitation). + */ +export function mergeMap(project: Project): Operator { + return mapOperator("mergeMap", project, "merge"); +} + +/** flatMap: alias of {@link mergeMap} (RxJS naming parity). */ +export function flatMap(project: Project): Operator { + return mapOperator("flatMap", project, "merge"); +} + +/** + * concatMap: project each source value to an inner, but run AT MOST ONE inner at a time — queue + * later source values (lazy projection) and activate the next only when the active inner COMPLETEs. + * Preserves source order. + */ +export function concatMap(project: Project): Operator { + return mapOperator("concatMap", project, "concat"); +} + +/** + * exhaustMap: while an inner is active, DROP new source values; project the next source value only + * after the active inner COMPLETEs. + */ +export function exhaustMap(project: Project): Operator { + return mapOperator("exhaustMap", project, "exhaust"); +} diff --git a/packages/ts/src/graph/operators.ts b/packages/ts/src/graph/operators.ts index a0c90aef..16b2456a 100644 --- a/packages/ts/src/graph/operators.ts +++ b/packages/ts/src/graph/operators.ts @@ -66,7 +66,11 @@ export function initNode( ctx.down([["ERROR", e]]); // D30: value-level throw → ERROR } }; - return new Node([...deps], body, { ...op.opts, ...opts }); + // D43-reserved / D51: stamp the operator's real factory onto the bare node so a runtime *Map + // inner (created here via fromAny, NOT registered in any graph) is named in describe's + // auto-discovery. A graph-bound g.initNode also records it in `_entries` (entry.factory wins + // there); this field is only read for a node absent from the graph index. Caller opts win. + return new Node([...deps], body, { factory: op.factory, ...op.opts, ...opts }); } /** map: emit fn(value). */ diff --git a/packages/ts/src/index.ts b/packages/ts/src/index.ts index 62947630..3e4a16df 100644 --- a/packages/ts/src/index.ts +++ b/packages/ts/src/index.ts @@ -7,7 +7,7 @@ */ export { type BatchCtx, batch } from "./batch/batch.js"; -export type { Ctx, CtxState, DepRecord, NodeFn, Sink } from "./ctx/types.js"; +export type { Ctx, CtxState, DepRecord, NodeFn, RewireNext, Sink } from "./ctx/types.js"; export { Dispatcher, defaultDispatcher, @@ -31,6 +31,14 @@ export { StateNode, type SugarOpts, } from "./graph/graph.js"; +export { + concatMap, + exhaustMap, + flatMap, + mergeMap, + type Project, + switchMap, +} from "./graph/higher-order.js"; export type { NodeProfile, ObserveEvent, ObserveStream, Profile } from "./graph/inspect.js"; export { distinctUntilChanged, diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index 5405edee..2f1288dc 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -14,6 +14,7 @@ */ import { currentBatch, deferToBatch } from "../batch/batch.js"; +import { deferRewire, enterWave, exitWave } from "../batch/boundary.js"; import type { Ctx, CtxState, DepRecord, NodeFn, Sink } from "../ctx/types.js"; import { type Dispatcher, defaultDispatcher, type Handle } from "../dispatcher/index.js"; import { @@ -60,8 +61,22 @@ export interface NodeOptions { dispatcher?: Dispatcher; /** Optional debug name (graph layer owns real naming/inspection). */ name?: string; + /** + * Real operator/source factory name for a STANDALONE graph-less node (D43-reserved; D51). + * The graph index (`_entries`) carries the factory for g.*-registered nodes, so this is only + * read for a node NOT in any graph index — a runtime *Map inner (bare `fromAny`/`initNode` + * node) auto-discovered by `describe` (R-describe / R-edges-derived / D51). Off the canonical + * wave path (R-node-thin intact — a pure annotation, never touched by the wave machinery). + */ + factory?: string; } +/** A queued deferred self-rewire op (R-rewire-deferred / D47), drained at the wave boundary. */ +type RewireOp = + | { kind: "add"; dep: Node; fn: NodeFn } + | { kind: "remove"; dep: Node; fn: NodeFn } + | { kind: "set"; deps: Node[]; fn: NodeFn }; + export class Node { private _deps: Node[]; private _handle: Handle | null; @@ -77,6 +92,8 @@ export class Node { private readonly _replayN: number; private readonly _dynamic: boolean; readonly name?: string; + /** R-describe/D51: real factory name for a standalone graph-less node (a runtime *Map inner). */ + readonly factory?: string; private _subscribers = new Set(); private _activated = false; @@ -150,6 +167,7 @@ export class Node { this._dynamic = opts.dynamic ?? false; this._pool = opts.pool ?? "sync"; this.name = opts.name; + this.factory = opts.factory; if (handleOrFn === null) this._handle = null; else if (typeof handleOrFn === "function") @@ -187,6 +205,16 @@ export class Node { return this._status; } + /** + * The node's CURRENT/LIVE deps (R-describe / R-edges-derived / D51) — readonly view of the + * live `_deps`, which a rewire (C-8 / C-11) mutates. The graph's describe() reads this (NOT a + * construction-time snapshot) so every edge corresponds to a real current subscription (D3). + * Inspection-only, like cache/status; never triggers computation. + */ + get deps(): readonly Node[] { + return this._deps; + } + /** * The fn handle (pure data `(poolId, handleId)`, D7) or null for state/passthrough * nodes. Inspection-only (L1.6 handle is referenceable/inspectable) — lets the graph @@ -199,43 +227,94 @@ export class Node { /** R-push-subscribe: a new sink receives START, then cached DATA (or DIRTY if dirty). */ subscribe(sink: Sink): () => void { - // R-terminal: late subscribe to a terminal node either resets (resubscribable) - // or is rejected (non-resubscribable, R2.2.7.b). - if (this._terminal !== undefined) { - if (this._resubscribable) this._resetLifecycle(); - else - throw new Error( - "subscribe: node is non-resubscribable and has terminated; the stream is permanently over (R-terminal / R2.2.7.b)", - ); - } + // Wave-owner boundary (R-rewire-deferred / D47): the activation cascade can run fns that + // issue ctx.rewireNext; the OUTERMOST exit drains them. Nested subscribes (dep wiring) + // just inc/dec the depth. + enterWave(); + try { + // R-terminal: late subscribe to a terminal node either resets (resubscribable) + // or is rejected (non-resubscribable, R2.2.7.b). + if (this._terminal !== undefined) { + if (this._resubscribable) this._resetLifecycle(); + else + throw new Error( + "subscribe: node is non-resubscribable and has terminated; the stream is permanently over (R-terminal / R2.2.7.b)", + ); + } - this._subscribers.add(sink); - sink(["START"]); - if (this._replayN > 0 && this._replayRing.length > 0) { - // R-replay-buffer: late subscriber gets the last N DATA after START. - for (const v of this._replayRing) sink(["DATA", v]); - } else if (this._hasData) { - sink(["DATA", this._cache]); - } else if (this._status === "dirty") { - sink(["DIRTY"]); - } + this._subscribers.add(sink); + sink(["START"]); + if (this._replayN > 0 && this._replayRing.length > 0) { + // R-replay-buffer: late subscriber gets the last N DATA after START. + for (const v of this._replayRing) sink(["DATA", v]); + } else if (this._hasData) { + sink(["DATA", this._cache]); + } else if (this._status === "dirty") { + sink(["DIRTY"]); + } - if (!this._activated) this._activate(); + if (!this._activated) this._activate(); - return () => { - if (!this._subscribers.delete(sink)) return; - if (this._subscribers.size === 0) this._deactivate(); - }; + return () => { + if (!this._subscribers.delete(sink)) return; + if (this._subscribers.size === 0) this._deactivate(); + }; + } finally { + exitWave(); + } } /** External emission toward sinks (state-node push, or async late-emit). One call = one wave. */ down(msgs: Wave): void { - this._down(msgs); + // Wave-owner boundary (D47): a state.set / external push that drives a fn issuing + // ctx.rewireNext drains at this outermost exit. + enterWave(); + try { + this._down(msgs); + } finally { + exitWave(); + } } /** Emit upstream toward deps — control tiers only (R-ctx-up). */ up(msgs: Wave): void { - this._up(msgs); + // Wave-owner boundary (D47). Internal dep-forwarding calls dep.up() nest under this. + enterWave(); + try { + this._up(msgs); + } finally { + exitWave(); + } + } + + // ── deferred self-rewire (R-rewire-deferred / D47): ctx.rewireNext drain support ── + + /** + * Enqueue a deferred self-rewire op (issued from this node's fn via `ctx.rewireNext`). + * Applied at the committed wave boundary (boundary.ts drain), never in place — the in-fn + * immediate path (`addDep`/`setDeps`/`removeDep`) still throws mid-run (D37/R-reentrancy). + */ + private _requestRewireNext(op: RewireOp): void { + deferRewire(() => this._applyRewireNext(op)); + } + + /** Apply one queued self-rewire at the boundary (drain thunk). */ + private _applyRewireNext(op: RewireOp): void { + // Terminal discards the pending queue (R-rewire-deferred): a node that went terminal + // during the wave drops its queued self-rewires. + if (this._terminal !== undefined) return; + try { + if (op.kind === "add") this.addDep(op.dep, op.fn); + else if (op.kind === "remove") this.removeDep(op.dep, op.fn); + else this.setDeps(op.deps, op.fn); + } catch (e) { + // An invalid deferred op (cycle / self / non-resubscribable terminal dep) surfaces as + // an ERROR on this node (D30-consistent) rather than stranding the rest of the drain + // queue. Reachable only on misuse — higher-order operator inners are fresh, acyclic + // leaf sources. Coerce a SENTINEL reason (a rewire fn that `throw undefined`s) to a real + // Error so _down's R-data-payload guard does not itself throw out of the drain. + this._down([["ERROR", e === undefined ? new Error("rewireNext op failed") : e]]); + } } // ── rewire (R-rewire / D42): intra-graph runtime topology mutation ── @@ -546,20 +625,32 @@ export class Node { if (t === "COMPLETE") { this._depTerminal[idx] = true; + // R-terminal-settles-dirty (B35): a terminal RELEASES this dep's outstanding in-wave + // DIRTY contribution (the exactly-one-settle invariant) — exactly as DATA/RESOLVED/ + // INVALIDATE do (a dirty-then-terminal-without-DATA dep would otherwise strand _pending + // and wedge the node, the deadlock R-invalidate-idempotent prevents for INVALIDATE). + this._releaseDepDirty(idx); if (this._completeWhenDepsComplete && this._allDepsComplete()) { - this._down([["COMPLETE"]]); + this._down([["COMPLETE"]]); // auto-cascade → node itself terminal (_pending moot) } else if (this._terminalAsRealInput) { - this._maybeRun(); + this._maybeRun(); // rescue/reduce/*Map: the fn reads ctx.depRecords[idx].terminal + } else { + // absorbed terminal, NOT an input: the dep's signalled change did not materialise + // (no DATA) → un-dirty downstream, keep cache (R-resolved-undirty balance). + this._settleAfterAbsorbedTerminal(); } return; } if (t === "ERROR") { this._depTerminal[idx] = msg[1]; + this._releaseDepDirty(idx); // R-terminal-settles-dirty (B35), as COMPLETE above if (this._errorWhenDepsError) { - this._down([["ERROR", msg[1]]]); + this._down([["ERROR", msg[1]]]); // auto-cascade → node itself terminal } else if (this._terminalAsRealInput) { - this._maybeRun(); + this._maybeRun(); // rescue/catch: the fn reads the dep's terminal (the error) + } else { + this._settleAfterAbsorbedTerminal(); } return; } @@ -608,6 +699,51 @@ export class Node { // paused via its own up() (lockset), not by an upstream dep. } + /** + * R-terminal-settles-dirty (B35): release a dep's outstanding in-wave DIRTY contribution. + * A settle-class event for that dep (DATA/RESOLVED inline above, INVALIDATE, and now + * COMPLETE/ERROR) clears its dirty flag + decrements _pending. No-op if the dep already + * settled this wave (DATA/RESOLVED ran first) — so the normal DATA-then-COMPLETE flow is + * unaffected. This makes the exactly-one-settle invariant a single, shared step. + */ + private _releaseDepDirty(idx: number): void { + if (this._depDirty[idx]) { + this._depDirty[idx] = false; + this._pending--; + } + } + + /** + * R-terminal-settles-dirty (B35): settle a node whose dirtied dep was released by an ABSORBED + * terminal that is NOT a real input (a plain derived/effect — the common default-node case when + * one of several deps completes while others stay live). Runs only when the release drained + * _pending while the node still owes a downstream settle (it broadcast DIRTY this wave): + * - if some OTHER dep delivered real DATA this wave (a value occurred) → recompute (→ DATA); + * - else the terminal's signalled change did not materialise (no value) → one undirty RESOLVED + * (R-resolved-undirty), keeping the cache (a terminal, unlike INVALIDATE, leaves the value). + * A terminalAsRealInput node instead recomputes unconditionally (its fn reads the terminal). + */ + private _settleAfterAbsorbedTerminal(): void { + if (this._pending !== 0 || !this._emittedDirtyThisWave) return; + // A real value occurred this wave (some OTHER dep delivered DATA) → recompute. _maybeRun + // runs the fn ONLY if it's not gated (first-run gate open, not paused); it may emit DATA, a + // fn-synthesized undirty RESOLVED, or nothing (gated / gate still holds). + const sawData = this._depBatch.some((b) => b !== null && b.length > 0); + if (sawData) this._maybeRun(); + // If after that the node STILL owes a downstream settle (no DATA occurred, OR the recompute + // was gated — e.g. the first-run gate holds because the terminated dep never delivered and + // terminalAsRealInput is false), balance the broadcast DIRTY with one undirty RESOLVED + // (R-resolved-undirty), keeping the cache (a terminal, unlike INVALIDATE, leaves the value). + // Without this fallback a DIRTY-then-terminal-without-DATA dep on a pre-first-run multi-dep + // node would strand the DIRTY → downstream wedged (the B35 class, in the gate-holds corner). + // Bare emit mirrors the INVALIDATE receive-arm; terminal×pause/batch coalescing = backlog B39. + if (this._emittedDirtyThisWave) { + this._emittedDirtyThisWave = false; + this._status = this._hasData ? "resolved" : "sentinel"; + this._emitToSubs(["RESOLVED"]); + } + } + private _markDirty(): void { this._status = "dirty"; if (!this._emittedDirtyThisWave) { @@ -776,8 +912,25 @@ export class Node { private _makeCtx(depRecords: readonly DepRecord[]): Ctx { const ctx: Ctx = { - up: (msgs) => this._up(msgs), - down: (msgs) => this._down(msgs), + // Wave-owner boundary (D47): a SYNC fn's emit nests under the public entry that drove + // it (cheap inc/dec, no early drain); an ASYNC-pool fn re-enters here from its stashed + // ctx at depth 0, so this is the boundary that drains any rewireNext it issued. + up: (msgs) => { + enterWave(); + try { + this._up(msgs); + } finally { + exitWave(); + } + }, + down: (msgs) => { + enterWave(); + try { + this._down(msgs); + } finally { + exitWave(); + } + }, depRecords, state: this._makeState(), onDeactivation: (fn) => { @@ -786,6 +939,12 @@ export class Node { onInvalidate: (fn) => { this._onInvalidate.push(fn); }, + // R-rewire-deferred (D47): defer a self-dep-set mutation to the committed boundary. + rewireNext: { + addDep: (dep, fn) => this._requestRewireNext({ kind: "add", dep, fn }), + removeDep: (dep, fn) => this._requestRewireNext({ kind: "remove", dep, fn }), + setDeps: (deps, fn) => this._requestRewireNext({ kind: "set", deps, fn }), + }, }; if (this._dynamic) { // R-dynamic-node: read a dep's latest by index. Untracked deps still drive waves and From ed78645b43700b119db077f53b6778082d60bfb5 Mon Sep 17 00:00:00 2001 From: David Chen Date: Sat, 30 May 2026 20:58:13 -0700 Subject: [PATCH 011/175] feat(ts): add combinators and time operators to enhance graph functionality - Introduced new combinators: `combine`, `combineLatest`, `withLatestFrom`, `zip`, `concat`, and `race` to facilitate multi-source data handling. - Added time operators: `delay`, `debounce`, `debounceTime`, `throttle`, and `throttleTime` for wall-clock time management in data streams. - Updated `index.ts` to export new combinators and time operators, improving usability and type management. - Created comprehensive tests for the new combinators and time operators, ensuring correct behavior and compliance with the updated functionality. - Enhanced the `Node` class to support the new operators and maintain proper state during operations. --- packages/ts/src/__tests__/combinators.test.ts | 199 ++++++++ packages/ts/src/__tests__/conformance.test.ts | 45 ++ .../ts/src/__tests__/higher-order.test.ts | 58 +++ packages/ts/src/__tests__/operators.test.ts | 302 +++++++++++- packages/ts/src/__tests__/time.test.ts | 78 ++++ packages/ts/src/graph/combinators.ts | 367 +++++++++++++++ packages/ts/src/graph/higher-order.ts | 79 ++++ packages/ts/src/graph/operators.ts | 440 ++++++++++++++++++ packages/ts/src/graph/time.ts | 104 +++++ packages/ts/src/index.ts | 38 ++ packages/ts/src/node/node.ts | 101 ++-- packages/ts/src/protocol/messages.ts | 10 + 12 files changed, 1791 insertions(+), 30 deletions(-) create mode 100644 packages/ts/src/__tests__/combinators.test.ts create mode 100644 packages/ts/src/__tests__/time.test.ts create mode 100644 packages/ts/src/graph/combinators.ts create mode 100644 packages/ts/src/graph/time.ts diff --git a/packages/ts/src/__tests__/combinators.test.ts b/packages/ts/src/__tests__/combinators.test.ts new file mode 100644 index 00000000..58410afd --- /dev/null +++ b/packages/ts/src/__tests__/combinators.test.ts @@ -0,0 +1,199 @@ +import { describe, expect, it } from "vitest"; +import { + buffer, + bufferCount, + combine, + combineLatest, + concat, + fromIter, + graph, + type Message, + race, + sample, + takeUntil, + withLatestFrom, + zip, +} from "../index.js"; + +const data = (msgs: Message[]) => + msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); +const lastType = (msgs: Message[]) => msgs[msgs.length - 1]?.[0]; + +// CSP-2.7 Slice 2 (D40 / D45): multi-source combinators + notifier-driven operators as +// declared-dep nodes (NOT the frozen reference's banned internal-subscribe islands). +describe("Slice 2 — multi-source combinators (CSP-2.7 / D45)", () => { + it("combine emits a tuple of latest values when any dep updates", () => { + const g = graph(); + const a = g.state(1); + const b = g.state("x"); + const c = g.initNode(combine<[number, string]>(), [a, b]); + const msgs: Message[] = []; + c.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([[1, "x"]]); + a.set(2); + b.set("y"); + expect(data(msgs)).toEqual([ + [1, "x"], + [2, "x"], + [2, "y"], + ]); + }); + + it("combineLatest is the combine alias", () => { + expect(combineLatest).toBe(combine); + }); + + it("withLatestFrom pairs primary with the latest secondary (initial pair not dropped)", () => { + const g = graph(); + const p = g.state(1); + const s = g.state("a"); + const w = g.initNode(withLatestFrom(), [p, s]); + const msgs: Message[] = []; + w.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([[1, "a"]]); // W1 fix — initial pair emitted + p.set(2); + s.set("b"); // secondary-only → no emit + p.set(3); + expect(data(msgs)).toEqual([ + [1, "a"], + [2, "a"], + [3, "b"], + ]); + }); + + it("zip combines one value from each dep in lockstep, completes with the shorter", () => { + const g = graph(); + const a = g.initNode(fromIter([1, 2, 3]), []); + const b = g.initNode(fromIter(["a", "b"]), []); + const z = g.initNode(zip<[number, string]>(), [a, b]); + const msgs: Message[] = []; + z.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([ + [1, "a"], + [2, "b"], + ]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("concat plays all of dep 0 then all of dep 1, then COMPLETE", () => { + const g = graph(); + const a = g.initNode(fromIter([1, 2]), []); + const b = g.initNode(fromIter([3, 4]), []); + const c = g.initNode(concat(), [a, b]); + const msgs: Message[] = []; + c.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([1, 2, 3, 4]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("race forwards only the first dep to deliver DATA", () => { + const g = graph(); + const a = g.initNode(fromIter([1, 2]), []); + const b = g.initNode(fromIter([10, 20]), []); + const r = g.initNode(race(), [a, b]); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([1, 2]); // a subscribed first → wins; b ignored + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("race (n>=3): a live dep can still win after others terminate empty (QA: no double-count)", () => { + // Regression for the cross-wave terminal double-count: with 3 deps, two completing empty in + // separate waves must NOT prematurely COMPLETE the race while the 3rd is still live. + const g = graph(); + const a = g.node([], null); // manual sources, silent until .down + const b = g.node([], null); + const c = g.node([], null); + const r = g.initNode(race(), [a, b, c]); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + a.down([["COMPLETE"]]); // 1 terminal + b.down([["COMPLETE"]]); // 2 terminal — OLD code: 1+2=3>=n → premature COMPLETE + expect(r.status).not.toBe("completed"); // c still live → race open + c.down([["DATA", 99]]); // c wins + c.down([["DATA", 100]]); + expect(data(msgs)).toEqual([99, 100]); + c.down([["COMPLETE"]]); // winner terminal → COMPLETE + expect(r.status).toBe("completed"); + }); + + it("race COMPLETEs when EVERY dep terminates without any DATA", () => { + const g = graph(); + const a = g.node([], null); + const b = g.node([], null); + const c = g.node([], null); + const r = g.initNode(race(), [a, b, c]); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + a.down([["COMPLETE"]]); + b.down([["COMPLETE"]]); + expect(r.status).not.toBe("completed"); + c.down([["COMPLETE"]]); // all terminal, no winner → COMPLETE + expect(data(msgs)).toEqual([]); + expect(r.status).toBe("completed"); + }); +}); + +describe("Slice 2 — notifier-driven (D46 first-cut)", () => { + it("buffer flushes accumulated source DATA on each notifier signal", () => { + const g = graph(); + const src = g.state(1); + const notify = g.state(false); + const b = g.initNode(buffer(), [src, notify]); + const msgs: Message[] = []; + b.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([[1]]); // notifier's initial flushes [1] + src.set(2); + src.set(3); + notify.set(true); + expect(data(msgs)).toEqual([[1], [2, 3]]); + }); + + it("bufferCount batches into fixed-size arrays, flushes remainder on COMPLETE", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 2, 3, 4, 5]), []); + const b = g.initNode(bufferCount(2), [src]); + const msgs: Message[] = []; + b.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([[1, 2], [3, 4], [5]]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("sample emits the latest source value on each notifier signal", () => { + const g = graph(); + const src = g.state(1); + const notify = g.state(false); + const s = g.initNode(sample(), [src, notify]); + const msgs: Message[] = []; + s.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([1]); // notifier initial samples the latest source (1) + src.set(2); + notify.set(true); + expect(data(msgs)).toEqual([1, 2]); + }); + + it("takeUntil forwards source DATA until the notifier delivers, then COMPLETE", () => { + const g = graph(); + const src = g.state(1); + const stop = g.node([]); // silent until .down (no initial → SENTINEL) + const tu = g.initNode(takeUntil(), [src, stop]); + const msgs: Message[] = []; + tu.subscribe((m) => msgs.push(m)); + src.set(2); + stop.down([["DATA", true]]); // notifier fires → stop + src.set(3); // terminated, ignored + expect(data(msgs)).toEqual([1, 2]); + expect(tu.status).toBe("completed"); + }); + + it("describe shows the combinator factory names (D6)", () => { + const g = graph(); + const a = g.state(1, { name: "a" }); + const b = g.state(2, { name: "b" }); + g.initNode(combine<[number, number]>(), [a, b], { name: "c" }); + g.initNode(zip<[number, number]>(), [a, b], { name: "z" }); + const byId = Object.fromEntries(g.describe().nodes.map((n) => [n.id, n])); + expect(byId.c.factory).toBe("combine"); + expect(byId.z.factory).toBe("zip"); + }); +}); diff --git a/packages/ts/src/__tests__/conformance.test.ts b/packages/ts/src/__tests__/conformance.test.ts index 0fb8b513..d89c8369 100644 --- a/packages/ts/src/__tests__/conformance.test.ts +++ b/packages/ts/src/__tests__/conformance.test.ts @@ -835,3 +835,48 @@ describe("C-15 a dep's terminal releases its in-wave DIRTY contribution (R-termi expect(d.cache).toBeUndefined(); // never produced a value }); }); + +// C-17 (R-deps-terminal / B42): an ABSORBED error (errorWhenDepsError:false) counts as TERMINAL for +// the completeWhenDepsComplete auto-COMPLETE cascade — order-independent (whichever terminal lands +// last fires it). The COMPLETION analogue of C-15's DIRTY-release. NOT hit by the landed catalog +// (rescue/race/sample use completeWhenDepsComplete:false) — a pure latent-wedge fix. +describe("C-17 — absorbed-error dep counts as terminal for auto-COMPLETE (B42 / R-deps-terminal)", () => { + const fwd: NodeFn = (ctx) => { + for (const r of ctx.depRecords) if (r.batch) for (const v of r.batch) ctx.down([["DATA", v]]); + }; + + it("(a) error-then-complete: C errors (absorbed), then B completes → D auto-COMPLETEs", () => { + const g = graph(); + const b = g.node([], null); // manual source + const c = g.node([], null); + const d = g.node([b, c], fwd, { errorWhenDepsError: false }); // absorbs C's error, stays live + collect(d); + c.down([["ERROR", new Error("boom")]]); // absorbed; B still live → not yet all-terminal + expect(d.status).not.toBe("completed"); + expect(d.status).not.toBe("errored"); // errorWhenDepsError:false → no auto-ERROR + b.down([["DATA", 1], ["COMPLETE"]]); // B terminal → ALL deps terminal (C errored) → COMPLETE + expect(d.status).toBe("completed"); + }); + + it("(b) complete-then-error: B completes, then C errors LAST → D still auto-COMPLETEs (ERROR-arm mirror)", () => { + const g = graph(); + const b = g.node([], null); + const c = g.node([], null); + const d = g.node([b, c], fwd, { errorWhenDepsError: false }); + collect(d); + b.down([["DATA", 1], ["COMPLETE"]]); // B terminal; C still live → not yet all-terminal + expect(d.status).not.toBe("completed"); + c.down([["ERROR", new Error("boom")]]); // C absorbed-error LANDS LAST → all terminal → COMPLETE + expect(d.status).toBe("completed"); // the ERROR-absorbed arm mirrors the COMPLETE arm (B42) + }); + + it("(c) errorWhenDepsError:true (default) auto-ERRORs on a dep error (absorbed path gated off)", () => { + const g = graph(); + const b = g.node([], null); + const c = g.node([], null); + const d = g.node([b, c], fwd); // defaults: errorWhenDepsError:true + collect(d); + c.down([["ERROR", new Error("boom")]]); // auto-cascade → ERROR before any complete-check + expect(d.status).toBe("errored"); + }); +}); diff --git a/packages/ts/src/__tests__/higher-order.test.ts b/packages/ts/src/__tests__/higher-order.test.ts index a6651c66..7f308c13 100644 --- a/packages/ts/src/__tests__/higher-order.test.ts +++ b/packages/ts/src/__tests__/higher-order.test.ts @@ -16,6 +16,7 @@ import { mergeMap, node, of, + repeat, switchMap, } from "../index.js"; @@ -229,6 +230,63 @@ describe("exhaustMap (D47)", () => { }); }); +describe("repeat (D47 self-rewire, factory-based — clean-slate complete design)", () => { + it("plays a fresh factory()-minted source `count` times in sequence, then COMPLETE", () => { + const g = graph(); + // factory mints a FRESH source each round (clean-slate hot nodes can't be re-run in place); + // `() => [1, 2]` → fromAny(iter) → fromIter([1,2]) → emits 1,2,COMPLETE per round. + const r = g.initNode( + repeat(() => [1, 2], 3), + [], + ); + const { msgs } = collect(r); + expect(data(msgs)).toEqual([1, 2, 1, 2, 1, 2]); + expect(msgs[msgs.length - 1][0]).toBe("COMPLETE"); + expect(r.status).toBe("completed"); + }); + + it("count=1 plays the source exactly once", () => { + const g = graph(); + const r = g.initNode( + repeat(() => [7], 1), + [], + ); + const { msgs } = collect(r); + expect(data(msgs)).toEqual([7]); + expect(r.status).toBe("completed"); + }); + + it("an inner ERROR aborts repeat (errorWhenDepsError)", () => { + const g = graph(); + // round 1 errors via a throwing projector inner (a derived that throws → D30 ERROR). + const r = g.initNode( + repeat(() => { + const gg = graph(); + const s = gg.state(0); + return gg.derived([s], () => { + throw new Error("inner boom"); + }); + }, 3), + [], + ); + const { msgs } = collect(r); + expect(r.status).toBe("errored"); + expect(msgs.some((m) => m[0] === "ERROR")).toBe(true); + }); + + it("describe shows repeat's real factory name + its live inner (D6/D51)", () => { + const g = graph(); + g.initNode( + repeat(() => [1], 2), + [], + { name: "rep" }, + ); + const snap = g.describe(); + const rep = snap.nodes.find((dn) => dn.id === "rep"); + expect(rep?.factory).toBe("repeat"); + }); +}); + describe("higher-order — describe (D39 / D51): real factory name + LIVE inner topology", () => { it("records the operator's real factory name; before any source value, only the construction edge", () => { const a = subject(); diff --git a/packages/ts/src/__tests__/operators.test.ts b/packages/ts/src/__tests__/operators.test.ts index 0fa1df64..9b7571a9 100644 --- a/packages/ts/src/__tests__/operators.test.ts +++ b/packages/ts/src/__tests__/operators.test.ts @@ -1,9 +1,34 @@ -import { describe, expect, it } from "vitest"; +import { describe, expect, it, vi } from "vitest"; import type { Message } from "../index.js"; -import { distinctUntilChanged, filter, graph, initNode, map, merge, scan, take } from "../index.js"; +import { + catchError, + distinctUntilChanged, + elementAt, + filter, + find, + first, + fromIter, + graph, + initNode, + last, + map, + merge, + onFirstData, + pairwise, + reduce, + rescue, + scan, + settle, + skip, + take, + takeWhile, + tap, + valve, +} from "../index.js"; const data = (msgs: Message[]) => msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); +const lastType = (msgs: Message[]) => msgs[msgs.length - 1]?.[0]; // D43: operators are free-standing factory definitions instantiated via the generic // g.initNode funnel (config folded into the factory; fn params annotated). describe still @@ -132,3 +157,276 @@ describe("core operators (free-standing factories via g.initNode, D43/D6)", () = expect(data(msgs)).toEqual([10, 12]); }); }); + +// CSP-2.7 catalog re-derive (D40): Slice 1 (single-dep transform/take/control) + Slice 3 +// (error-handling). Per-language sugar (D6/D24, never in parity). Terminal-emitting operators read +// ctx.depRecords[0].terminal (R-deps-terminal). D49: every occurrence is DATA; a no-emit wave → +// substrate-synthesized undirty RESOLVED. +describe("Slice 1 — single-dep transform/take/control (CSP-2.7)", () => { + it("reduce emits the final accumulator on source COMPLETE", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 2, 3]), []); + const r = g.initNode( + reduce((acc: number, v: number) => acc + v, 0), + [src], + ); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([6]); + expect(lastType(msgs)).toBe("COMPLETE"); + expect(r.status).toBe("completed"); + }); + + it("reduce on an empty source emits the seed", () => { + const g = graph(); + const empty = g.initNode(fromIter([]), []); + const r = g.initNode( + reduce((acc: number, v: number) => acc + v, 42), + [empty], + ); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([42]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("pairwise emits consecutive [prev, curr]; first value produces no pair", () => { + const g = graph(); + const s = g.state(1); + const p = g.initNode(pairwise(), [s]); + const msgs: Message[] = []; + p.subscribe((m) => msgs.push(m)); + s.set(2); + s.set(3); + expect(data(msgs)).toEqual([ + [1, 2], + [2, 3], + ]); + }); + + it("skip drops the first n DATA", () => { + const g = graph(); + const s = g.state(1); + const sk = g.initNode(skip(2), [s]); + const msgs: Message[] = []; + sk.subscribe((m) => msgs.push(m)); + s.set(2); // skipped (count 1) + s.set(3); // emitted + s.set(4); // emitted + expect(data(msgs)).toEqual([3, 4]); + }); + + it("takeWhile emits while pred holds, then COMPLETE (non-inclusive)", () => { + const g = graph(); + const s = g.state(1); + const tw = g.initNode( + takeWhile((n: number) => n < 3), + [s], + ); + const msgs: Message[] = []; + tw.subscribe((m) => msgs.push(m)); + s.set(2); + s.set(3); // fails pred → COMPLETE, not emitted + s.set(4); // terminated, ignored + expect(data(msgs)).toEqual([1, 2]); + expect(tw.status).toBe("completed"); + }); + + it("first emits the first matching value then COMPLETE", () => { + const g = graph(); + const s = g.state(1); + const f = g.initNode( + first((n: number) => n > 2), + [s], + ); + const msgs: Message[] = []; + f.subscribe((m) => msgs.push(m)); + s.set(2); // no match + s.set(5); // first match + s.set(6); // terminated + expect(data(msgs)).toEqual([5]); + expect(f.status).toBe("completed"); + }); + + it("last emits the last matching value on COMPLETE", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 2, 3, 4]), []); + const l = g.initNode( + last((n: number) => n % 2 === 0), + [src], + ); + const msgs: Message[] = []; + l.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([4]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("find emits the first matching value then COMPLETE", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 2, 3, 4]), []); + const fd = g.initNode( + find((n: number) => n > 2), + [src], + ); + const msgs: Message[] = []; + fd.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([3]); + expect(fd.status).toBe("completed"); + }); + + it("find not-found → bare COMPLETE (no undefined emit, SENTINEL edge)", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 2]), []); + const fd = g.initNode( + find((n: number) => n > 100), + [src], + ); + const msgs: Message[] = []; + fd.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("elementAt emits the value at the index then COMPLETE", () => { + const g = graph(); + const src = g.initNode(fromIter([10, 20, 30]), []); + const e = g.initNode(elementAt(1), [src]); + const msgs: Message[] = []; + e.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([20]); + expect(e.status).toBe("completed"); + }); + + it("tap (function form) runs the side effect and passes values through", () => { + const g = graph(); + const s = g.state(1); + const spy = vi.fn(); + const t = g.initNode(tap(spy), [s]); + const msgs: Message[] = []; + t.subscribe((m) => msgs.push(m)); + s.set(2); + expect(data(msgs)).toEqual([1, 2]); + expect(spy.mock.calls).toEqual([[1], [2]]); + }); + + it("tap (observer form) observes data + complete", () => { + const g = graph(); + const src = g.initNode(fromIter([7, 8]), []); + const onData = vi.fn(); + const onComplete = vi.fn(); + const t = g.initNode(tap({ data: onData, complete: onComplete }), [src]); + const msgs: Message[] = []; + t.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([7, 8]); + expect(onData.mock.calls).toEqual([[7], [8]]); + expect(onComplete).toHaveBeenCalledOnce(); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("onFirstData fires once on the first qualifying value", () => { + const g = graph(); + const s = g.state(1); + const spy = vi.fn(); + const o = g.initNode(onFirstData(spy), [s]); + const msgs: Message[] = []; + o.subscribe((m) => msgs.push(m)); + s.set(2); + s.set(3); + expect(data(msgs)).toEqual([1, 2, 3]); + expect(spy).toHaveBeenCalledOnce(); + expect(spy).toHaveBeenCalledWith(1); + }); + + it("settle forwards DATA and COMPLETEs after quietWaves of no change", () => { + const g = graph(); + const s = g.state(1); + const st = g.initNode(settle({ quietWaves: 2, equals: (a, b) => a === b }), [s]); + const msgs: Message[] = []; + st.subscribe((m) => msgs.push(m)); + s.set(1); // no change → quiet 1 + s.set(1); // no change → quiet 2 → COMPLETE + expect(data(msgs)).toEqual([1, 1, 1]); + expect(st.status).toBe("completed"); + }); + + it("describe shows the real catalog factory names (D6)", () => { + const g = graph(); + const s = g.state(0, { name: "s" }); + g.initNode(skip(1), [s], { name: "sk" }); + g.initNode(pairwise(), [s], { name: "pw" }); + const byId = Object.fromEntries(g.describe().nodes.map((n) => [n.id, n])); + expect(byId.sk.factory).toBe("skip"); + expect(byId.pw.factory).toBe("pairwise"); + }); +}); + +describe("Slice 3 — error-handling control (CSP-2.7)", () => { + // A source that ERRORs on a sentinel value (D30 throw→ERROR at the derived boundary). + const flakyChain = (g: ReturnType) => { + const s = g.state(1); + const bad = g.derived([s], (v: number) => { + if (v === 2) throw new Error("boom"); + return v * 10; + }); + return { s, bad }; + }; + + it("rescue replaces an upstream ERROR with a recovered value (stream stays live)", () => { + const g = graph(); + const { s, bad } = flakyChain(g); + const r = g.initNode( + rescue(() => -1), + [bad], + ); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([10]); + s.set(2); // bad throws → ERROR → rescue recovers → -1 + expect(data(msgs)).toEqual([10, -1]); + expect(r.status).not.toBe("errored"); + }); + + it("rescue forwards a normal source COMPLETE", () => { + const g = graph(); + const src = g.initNode(fromIter([1, 2]), []); + const r = g.initNode( + rescue(() => -1), + [src], + ); + const msgs: Message[] = []; + r.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([1, 2]); + expect(lastType(msgs)).toBe("COMPLETE"); + }); + + it("catchError is the rescue alias", () => { + expect(catchError).toBe(rescue); + }); + + it("valve gates DATA on a boolean control, re-emits on gate open", () => { + const g = graph(); + const src = g.state(1); + const open = g.state(true); + const v = g.initNode(valve(), [src, open]); + const msgs: Message[] = []; + v.subscribe((m) => msgs.push(m)); + expect(data(msgs)).toEqual([1]); // open at activation → forwards + open.set(false); // close + src.set(2); // gated out + expect(data(msgs)).toEqual([1]); + open.set(true); // re-open → re-emit last source value (2) + expect(data(msgs)).toEqual([1, 2]); + }); + + it("valve fires abortInFlight on the truthy→falsy edge only", () => { + const g = graph(); + const src = g.state(1); + const open = g.state(true); + const ctrl = new AbortController(); + const v = g.initNode(valve({ abortInFlight: ctrl }), [src, open]); + v.subscribe(() => {}); + expect(ctrl.signal.aborted).toBe(false); + open.set(false); // truthy→falsy edge → abort + expect(ctrl.signal.aborted).toBe(true); + }); +}); diff --git a/packages/ts/src/__tests__/time.test.ts b/packages/ts/src/__tests__/time.test.ts new file mode 100644 index 00000000..666a4b9f --- /dev/null +++ b/packages/ts/src/__tests__/time.test.ts @@ -0,0 +1,78 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { + debounce, + debounceTime, + delay, + graph, + type Message, + throttle, + throttleTime, +} from "../index.js"; + +const data = (msgs: Message[]) => + msgs.filter((m) => m[0] === "DATA").map((m) => (m as ["DATA", unknown])[1]); + +// CSP-2.7 Slice 4 (D52 / B23): wall-clock time operators as *Map + timer compositions — NO raw +// setTimeout in operator bodies (R-no-raw-async). Driven with fake timers like the source tests. +describe("Slice 4 — wall-clock time operators (D52, *Map + timer)", () => { + beforeEach(() => vi.useFakeTimers()); + afterEach(() => vi.useRealTimers()); + + it("delay shifts every value by ms, keeping all (mergeMap + timer)", () => { + const g = graph(); + const s = g.node([], null); // manual source + const d = g.initNode(delay(100), [s]); + const msgs: Message[] = []; + d.subscribe((m) => msgs.push(m)); + s.down([["DATA", 1]]); + s.down([["DATA", 2]]); + expect(data(msgs)).toEqual([]); // nothing yet — both delayed + vi.advanceTimersByTime(100); + expect(data(msgs)).toEqual([1, 2]); // both fire after 100ms, in order + }); + + it("debounce emits only the latest value after ms of quiet (switchMap + timer cancel-restart)", () => { + const g = graph(); + const s = g.node([], null); + const d = g.initNode(debounce(100), [s]); + const msgs: Message[] = []; + d.subscribe((m) => msgs.push(m)); + s.down([["DATA", 1]]); + vi.advanceTimersByTime(50); // 1's timer not yet fired + s.down([["DATA", 2]]); // cancels 1's timer (onDeactivation clearTimeout), restarts + vi.advanceTimersByTime(50); // 50ms since 2 — still not quiet enough + expect(data(msgs)).toEqual([]); + vi.advanceTimersByTime(50); // now 100ms since 2 → emit 2 (1 was dropped) + expect(data(msgs)).toEqual([2]); + }); + + it("debounceTime is the debounce alias (real factory name in describe)", () => { + const g = graph(); + const s = g.node([], null, { name: "s" }); + g.initNode(debounceTime(50), [s], { name: "db" }); + const byId = Object.fromEntries(g.describe().nodes.map((n) => [n.id, n])); + expect(byId.db.factory).toBe("debounceTime"); + }); + + it("throttle emits the leading value, then ignores the source for ms (exhaustMap + window)", () => { + const g = graph(); + const s = g.node([], null); + const t = g.initNode(throttle(100), [s]); + const msgs: Message[] = []; + t.subscribe((m) => msgs.push(m)); + s.down([["DATA", 1]]); // leading edge → emit 1 immediately + s.down([["DATA", 2]]); // within the window → dropped + expect(data(msgs)).toEqual([1]); + vi.advanceTimersByTime(100); // window closes + s.down([["DATA", 3]]); // new window → emit 3 + expect(data(msgs)).toEqual([1, 3]); + }); + + it("throttleTime is the throttle alias (real factory name)", () => { + const g = graph(); + const s = g.node([], null, { name: "s" }); + g.initNode(throttleTime(50), [s], { name: "th" }); + const byId = Object.fromEntries(g.describe().nodes.map((n) => [n.id, n])); + expect(byId.th.factory).toBe("throttleTime"); + }); +}); diff --git a/packages/ts/src/graph/combinators.ts b/packages/ts/src/graph/combinators.ts new file mode 100644 index 00000000..1e696616 --- /dev/null +++ b/packages/ts/src/graph/combinators.ts @@ -0,0 +1,367 @@ +/** + * Multi-source combinators + notifier-driven operators (CSP-2.7 catalog re-derive, D40 / D45). + * + * Per-language sugar (D6/D24, never in parity). Each forms topology ONLY via DECLARED deps (D45) — + * the frozen pure-ts reference (D41) builds these as a producer that calls `subscribeOr` on each + * source INSIDE the fn body, which is the D45-banned describe island (an edge `describe` cannot + * show). Re-derived here as static-dep nodes with `partial:true` (fire on any dep, R-first-run-gate + * off) + `ctx.state` (queue / phase / winner) — so every edge is a real subscription `describe` + * shows truthfully (D3 / R-edges-derived). Multi-source completion / error reads + * `ctx.depRecords[i].terminal` (R-deps-terminal) via `completeWhenDepsComplete:false + + * terminalAsRealInput:true`. + * + * Under D49 (no equals-absorption) `combine` emits a fresh tuple on EVERY contributing wave (no + * substrate tuple-dedup; the pure-ts `equals` tuple comparator is gone) — pair with + * `distinctUntilChanged` for opt-in dedup. SENTINEL ("a dep has never delivered DATA") is detected + * via `rec.latest === undefined` (R-sentinel: `undefined` is the global sentinel, `null` is a valid + * value). + * + * NOT re-derived in this cut (flagged): the window family (`window`/`windowCount`/`windowTime`) + * emits `Node>` nested sub-streams — the frozen reference creates child nodes inside the fn + * body and pushes DATA into them imperatively, which is a describe island AND an open question for + * how an emitted live sub-stream appears in `describe` (D45/D51). Deferred to a design pass. + */ + +import type { Ctx } from "../ctx/types.js"; +import type { Operator } from "./operators.js"; + +/** + * combine (alias `combineLatest`): emit a tuple of every dep's latest value whenever ANY dep + * delivers, once ALL deps have delivered at least one value. `partial:true`; the all-delivered gate + * is `rec.latest !== undefined` per dep. Completes when ALL deps complete (default + * completeWhenDepsComplete); any dep ERROR propagates (default errorWhenDepsError). Emits a fresh + * tuple per wave (D49 — no dedup; compose `distinctUntilChanged` to suppress unchanged tuples). + */ +export function combine(): Operator { + return { + factory: "combine", + opts: { partial: true }, + body: (ctx) => { + const vals = ctx.depRecords.map((r) => r.latest); + if (vals.some((v) => v === undefined)) return; // not all deps delivered → undirty RESOLVED + ctx.down([["DATA", vals as unknown as T]]); + }, + }; +} + +/** combineLatest: RxJS-named alias of {@link combine}. */ +export const combineLatest = combine; + +/** + * withLatestFrom: on each PRIMARY (dep 0) value, emit `[primary, latestSecondary]`; a secondary-only + * (dep 1) wave updates the cached secondary but emits nothing. Uses the DEFAULT first-run gate + * (`partial:false`), NOT partial — the frozen reference's Phase 10.5 W1 fix: with `partial:true` the + * primary's push-on-subscribe fires the fn BEFORE the secondary has delivered, so the initial pair + * is dropped. The gate holds the first run until BOTH deps settle, then fires once with both + * populated; subsequent waves are gate-free and fire on primary-alone. Completes when the PRIMARY + * completes (not the secondary) — `completeWhenDepsComplete:false + terminalAsRealInput:true` with a + * primary-terminal check; a secondary terminal is ignored (its last value persists). + */ +export function withLatestFrom(): Operator { + return { + factory: "withLatestFrom", + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const primary = ctx.depRecords[0]; + const secondary = ctx.depRecords[1]; + if (primary.terminal === true) { + ctx.down([["COMPLETE"]]); + return; + } + const b0 = primary.batch; + if (!b0 || b0.length === 0) return; // secondary-only wave → quiet + const sec = secondary.latest as B | undefined; + if (sec === undefined) return; // secondary never delivered → quiet + for (const v of b0) ctx.down([["DATA", [v, sec] as readonly [A, B]]]); + }, + }; +} + +/** Internal: per-dep FIFO queues for {@link zip}. */ +interface ZipState { + queues: unknown[][]; +} + +/** + * zip: combine one value from EACH dep, in lockstep, into a tuple — buffering per-dep queues until + * every dep has at least one queued value, then shifting one from each. `partial:true` + + * `ctx.state` queues. Completes as soon as ANY dep is terminal AND its queue is empty (no further + * tuple can form) — `completeWhenDepsComplete:false + terminalAsRealInput:true`. Any dep ERROR + * propagates (default errorWhenDepsError). + */ +export function zip(): Operator { + return { + factory: "zip", + opts: { partial: true, completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const n = ctx.depRecords.length; + const st = ctx.state.get() ?? { queues: Array.from({ length: n }, () => []) }; + for (let i = 0; i < n; i++) { + const b = ctx.depRecords[i].batch; + if (b) for (const v of b) st.queues[i].push(v); + } + while (st.queues.every((q) => q.length > 0)) { + ctx.down([["DATA", st.queues.map((q) => q.shift()) as unknown as T]]); + } + ctx.state.set(st); + // A terminal dep whose queue is now drained can never contribute another tuple → COMPLETE. + for (let i = 0; i < n; i++) { + if (ctx.depRecords[i].terminal === true && st.queues[i].length === 0) { + ctx.down([["COMPLETE"]]); + return; + } + } + }, + }; +} + +/** Internal: phase + buffered second-source values for {@link concat}. */ +interface ConcatState { + phase: 0 | 1; + pending: S[]; + secondDone: boolean; +} + +/** + * concat: play ALL of dep 0, then ALL of dep 1. Dep-1 DATA arriving during phase 0 is buffered and + * flushed at the handoff (dep 0 COMPLETE). `partial:true` + `ctx.state` phase. Completes when dep 1 + * completes (or when dep 0 completes and dep 1 already had). `completeWhenDepsComplete:false + + * terminalAsRealInput:true`; any dep ERROR propagates (default errorWhenDepsError). + */ +export function concat(): Operator { + return { + factory: "concat", + opts: { partial: true, completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const first = ctx.depRecords[0]; + const second = ctx.depRecords[1]; + const st: ConcatState = ctx.state.get>() ?? { + phase: 0, + pending: [], + secondDone: false, + }; + if (st.phase === 0) { + if (first.batch) for (const v of first.batch) ctx.down([["DATA", v]]); + if (second.batch) for (const v of second.batch) st.pending.push(v as S); + if (second.terminal === true) st.secondDone = true; + if (first.terminal === true) { + st.phase = 1; + for (const v of st.pending) ctx.down([["DATA", v]]); + st.pending = []; + if (st.secondDone) ctx.down([["COMPLETE"]]); + } + } else { + if (second.batch) for (const v of second.batch) ctx.down([["DATA", v]]); + if (second.terminal === true) ctx.down([["COMPLETE"]]); + } + ctx.state.set(st); + }, + }; +} + +/** Internal: which dep won the {@link race} (null until the first DATA). */ +interface RaceState { + winner: number | null; +} + +/** + * race: the FIRST dep to deliver DATA wins; thereafter only the winner's messages flow (losers are + * dropped, including their terminals). `partial:true` + `ctx.state` winner. `errorWhenDepsError:false` + * (a LOSER's error must not error the race — only the winner's) + `completeWhenDepsComplete:false + + * terminalAsRealInput:true`. If every dep terminates before any DATA, COMPLETE. + */ +export function race(): Operator { + return { + factory: "race", + opts: { + partial: true, + errorWhenDepsError: false, + completeWhenDepsComplete: false, + terminalAsRealInput: true, + }, + body: (ctx) => { + const n = ctx.depRecords.length; + const st: RaceState = ctx.state.get() ?? { winner: null }; + if (st.winner === null) { + // Find the first dep that delivered DATA this wave → it wins. + for (let i = 0; i < n; i++) { + const b = ctx.depRecords[i].batch; + if (b && b.length > 0) { + st.winner = i; + for (const v of b) ctx.down([["DATA", v]]); + break; + } + } + if (st.winner === null) { + // No winner yet — if EVERY dep is terminal (none ever delivered DATA), COMPLETE. + // Recompute from the (sticky, terminal-is-forever) flags each wave — NEVER + // accumulate a counter, or a dep that terminated in an earlier wave is re-counted + // every subsequent wave (premature COMPLETE while a later dep is still live, n>=3). + if (ctx.depRecords.every((r) => r.terminal !== undefined)) { + ctx.state.set(st); + ctx.down([["COMPLETE"]]); + return; + } + } + } else { + const w = ctx.depRecords[st.winner]; + if (w.batch) for (const v of w.batch) ctx.down([["DATA", v]]); + if (w.terminal === true) { + ctx.state.set(st); + ctx.down([["COMPLETE"]]); + return; + } + if (w.terminal !== undefined) { + ctx.state.set(st); + ctx.down([["ERROR", w.terminal]]); + return; + } + } + ctx.state.set(st); + }, + }; +} + +// ── notifier-driven (D46 first-cut: gated on a reactive notifier dep, no wall-clock) ── + +/** + * buffer: accumulate dep 0 (source) DATA; flush the buffer as an array each time dep 1 (notifier) + * delivers DATA. `partial:true` + `ctx.state` buffer. On source COMPLETE, flush any remainder then + * COMPLETE (`completeWhenDepsComplete:false + terminalAsRealInput:true`; a notifier terminal is + * ignored). Source/notifier ERROR propagates (default errorWhenDepsError). + */ +export function buffer(): Operator { + return { + factory: "buffer", + opts: { partial: true, completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const source = ctx.depRecords[0]; + const notifier = ctx.depRecords[1]; + const buf = ctx.state.get() ?? []; + if (source.batch) for (const v of source.batch) buf.push(v as S); + if (source.terminal === true) { + if (buf.length > 0) ctx.down([["DATA", [...buf]]]); + ctx.state.set([]); + ctx.down([["COMPLETE"]]); + return; + } + if (notifier.batch && notifier.batch.length > 0) { + ctx.down([["DATA", [...buf]]]); // flush (may be empty) on each notifier signal + ctx.state.set([]); + return; + } + ctx.state.set(buf); + }, + }; +} + +/** + * bufferCount: batch consecutive source DATA into arrays of length `count`; the remainder flushes on + * source COMPLETE. Single dep + `ctx.state` buffer + `completeWhenDepsComplete:false + + * terminalAsRealInput:true`. + */ +export function bufferCount(count: number): Operator { + if (!Number.isInteger(count) || count < 1) { + throw new RangeError(`bufferCount: count must be a positive integer (got ${count})`); + } + return { + factory: "bufferCount", + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const r = ctx.depRecords[0]; + const buf = ctx.state.get() ?? []; + if (r.batch) { + for (const v of r.batch) { + buf.push(v as S); + if (buf.length >= count) ctx.down([["DATA", buf.splice(0, buf.length)]]); + } + } + if (r.terminal === true) { + if (buf.length > 0) ctx.down([["DATA", [...buf]]]); + ctx.state.set([]); + ctx.down([["COMPLETE"]]); + return; + } + ctx.state.set(buf); + }, + }; +} + +/** Internal: latest sampled source value + whether the source has completed, for {@link sample}. */ +interface SampleState { + last: { v: S } | undefined; + sourceDone: boolean; +} + +/** + * sample: emit the source's (dep 0) most recent value each time the notifier (dep 1) delivers DATA. + * `partial:true` + `ctx.state`. The notifier completing completes the operator; the source completing + * stops sampling (clears the held value); an ERROR from either terminates. + * `completeWhenDepsComplete:false + errorWhenDepsError:false + terminalAsRealInput:true` (manual + * terminal handling). No flush of the held value on source COMPLETE (RxJS sample semantics). + */ +export function sample(): Operator { + return { + factory: "sample", + opts: { + partial: true, + completeWhenDepsComplete: false, + errorWhenDepsError: false, + terminalAsRealInput: true, + }, + body: (ctx) => { + const source = ctx.depRecords[0]; + const notifier = ctx.depRecords[1]; + const st: SampleState = ctx.state.get>() ?? { + last: undefined, + sourceDone: false, + }; + // ERROR from either dep → terminate. + if (source.terminal !== undefined && source.terminal !== true) { + ctx.down([["ERROR", source.terminal]]); + return; + } + if (notifier.terminal !== undefined && notifier.terminal !== true) { + ctx.down([["ERROR", notifier.terminal]]); + return; + } + if (source.batch) for (const v of source.batch) st.last = { v: v as S }; + if (source.terminal === true) { + st.sourceDone = true; + st.last = undefined; + } + if (notifier.terminal === true) { + ctx.state.set(st); + ctx.down([["COMPLETE"]]); + return; + } + if (notifier.batch && notifier.batch.length > 0 && st.last !== undefined && !st.sourceDone) { + ctx.down([["DATA", st.last.v]]); + } + ctx.state.set(st); + }, + }; +} + +/** + * takeUntil: forward dep 0 (source) DATA until dep 1 (notifier) delivers its first DATA, then + * COMPLETE. `partial:true` + `completeWhenDepsComplete:false + terminalAsRealInput:true`: a source + * COMPLETE forwards COMPLETE; a notifier DATA triggers COMPLETE. Source/notifier ERROR propagates + * (default errorWhenDepsError). + */ +export function takeUntil(): Operator { + return { + factory: "takeUntil", + opts: { partial: true, completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx: Ctx) => { + const source = ctx.depRecords[0]; + const notifier = ctx.depRecords[1]; + if (notifier.batch && notifier.batch.length > 0) { + ctx.down([["COMPLETE"]]); // notifier fired → stop + return; + } + if (source.batch) for (const v of source.batch) ctx.down([["DATA", v]]); + if (source.terminal === true) ctx.down([["COMPLETE"]]); + }, + }; +} diff --git a/packages/ts/src/graph/higher-order.ts b/packages/ts/src/graph/higher-order.ts index 27294e3b..eb50bdaa 100644 --- a/packages/ts/src/graph/higher-order.ts +++ b/packages/ts/src/graph/higher-order.ts @@ -185,3 +185,82 @@ export function concatMap(project: Project): Operator(project: Project): Operator { return mapOperator("exhaustMap", project, "exhaust"); } + +/** Per-node bookkeeping for {@link repeat}: the round index + the current live inner. */ +interface RepeatState { + started: boolean; + round: number; + inner: Node | null; +} + +/** + * repeat: play a source `count` times in sequence (RxJS `repeat`). A DEPLESS self-driving operator + * (Operator, instantiate via `g.initNode(repeat(factory, count), [])`) — NOT a dep-operator. + * + * It takes a `factory: () => NodeInput` (a RECIPE), not a source Node, because clean-slate Nodes + * are HOT/shared (multicast + cache): re-subscribing the SAME node replays its cache, it does not + * re-RUN the source. RxJS `repeat` re-subscribes the COLD source = a fresh subscription each round; + * the clean-slate analogue is a fresh node per round → a factory. (A `repeat(count)`-over-a-Node + * shape would need a substrate force-resubscribe affordance — D47's no-net-change-is-a-no-op makes + * `removeDep(S)+addDep(S)` on the SAME node cancel out — deferred to that substrate work.) + * + * Mechanism (reuses the D47 self-rewire substrate, like the *Map family; NOT an internal subscribe, + * D45): on activation it mints round 0's inner via `ctx.rewireNext.addDep`; each round's inner DATA + * is forwarded; on the inner's COMPLETE (`completeWhenDepsComplete:false + terminalAsRealInput:true`) + * it removeDep's the finished inner and, if rounds remain, addDep's a FRESH `factory()` inner (a + * distinct node → a real net change, not the no-op same-node case); after the last round it emits + * COMPLETE. An inner ERROR auto-forwards (errorWhenDepsError default → repeat errors). The body is + * SELF-CATCHING (D30) and re-supplied on every rewire. The factory MUST mint a FRESH node per call + * (a factory returning the same Node makes removeDep+addDep a net-zero no-op → repeat wedges). + */ +export function repeat(factory: () => NodeInput, count: number): Operator { + if (!Number.isInteger(count) || count < 1) { + throw new RangeError(`repeat: count must be a positive integer (got ${count})`); + } + const body: NodeFn = (ctx: Ctx) => { + try { + const st = (ctx.state.get() ?? { + started: false, + round: 0, + inner: null, + }) as RepeatState; + // 1. forward the current inner's DATA this wave (the inner is the sole dep once wired). + for (const r of ctx.depRecords) { + if (r.batch && r.batch.length > 0) for (const v of r.batch) ctx.down([["DATA", v]]); + } + // 2. activation: mint round 0's inner. + if (!st.started) { + st.started = true; + st.round = 0; + const inner = fromAny(factory(), { iter: true }) as Node; + st.inner = inner; + ctx.state.set(st); + ctx.rewireNext.addDep(inner, body); + return; + } + // 3. inner COMPLETE → next round or terminal COMPLETE. + if (st.inner !== null && ctx.depRecords[0]?.terminal === true) { + const old = st.inner; + ctx.rewireNext.removeDep(old, body); // bound the finished round's inner + if (st.round + 1 < count) { + st.round += 1; + const next = fromAny(factory(), { iter: true }) as Node; + st.inner = next; + ctx.state.set(st); + ctx.rewireNext.addDep(next, body); + } else { + st.inner = null; + ctx.state.set(st); + ctx.down([["COMPLETE"]]); + } + } + } catch (e) { + ctx.down([["ERROR", e]]); // D30 (self-catch survives rewire) + } + }; + return { + factory: "repeat", + body, + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + }; +} diff --git a/packages/ts/src/graph/operators.ts b/packages/ts/src/graph/operators.ts index 16b2456a..cf7e5fc3 100644 --- a/packages/ts/src/graph/operators.ts +++ b/packages/ts/src/graph/operators.ts @@ -73,6 +73,16 @@ export function initNode( return new Node([...deps], body, { factory: op.factory, ...op.opts, ...opts }); } +// ── Slice 1 — single-dep transform / take / control (CSP-2.7 catalog re-derive, D40) ── +// All read dep 0 positionally + emit via ctx.down. Under D49 every occurrence is DATA (no +// equals-absorption); a body that returns WITHOUT emitting gets a substrate-synthesized undirty +// RESOLVED (R-resolved-undirty) — so a "skip this wave" is a bare `return`. Terminal-emitting +// operators (reduce/last/find/elementAt) read `ctx.depRecords[0].terminal` via the +// `completeWhenDepsComplete:false + terminalAsRealInput:true` flags (R-deps-terminal): the fn fires +// on the source's COMPLETE and emits its final value. NOT 1:1 pure-ts ports — the frozen reference +// (D41) uses a producer + internal `subscribeOr`, the D45-banned describe island; these are +// declared-dep nodes whose single edge `describe` shows truthfully. + /** map: emit fn(value). */ export function map(fn: (v: S) => T): Operator { return { @@ -157,3 +167,433 @@ export function merge(): Operator { }, }; } + +/** + * reduce: accumulate over the whole source, emit ONE final value on the source's COMPLETE + * (RxJS `reduce`). Unlike {@link scan} (which emits every step), reduce stays quiet until the + * source terminates. `completeWhenDepsComplete:false + terminalAsRealInput:true`: the fn fires on + * the source COMPLETE (reading `terminal===true`) and emits the accumulator + COMPLETE. An empty + * source emits the seed (RxJS parity). A source ERROR auto-forwards (errorWhenDepsError default). + */ +export function reduce(reducer: (acc: T, v: S) => T, seed: T): Operator { + return { + factory: "reduce", + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const r = ctx.depRecords[0]; + let acc = ctx.state.get() ?? seed; + if (r.batch) for (const v of r.batch) acc = reducer(acc, v as S); + ctx.state.set(acc); + if (r.terminal === true) ctx.down([["DATA", acc], ["COMPLETE"]]); + // else (live DATA wave): no emit → substrate-synthesized undirty RESOLVED (D49). + }, + }; +} + +/** + * pairwise: emit `[previous, current]` for each consecutive pair; the very first value produces no + * pair (quiet → undirty RESOLVED). Previous value is kept in `ctx.state`. + */ +export function pairwise(): Operator { + return { + factory: "pairwise", + body: (ctx) => { + const v = ctx.depRecords[0].latest as S; + const st = ctx.state.get<{ prev: S }>(); + if (st) ctx.down([["DATA", [st.prev, v] as const]]); + ctx.state.set({ prev: v }); + }, + }; +} + +/** skip: drop the first `n` DATA values, then pass the rest through. */ +export function skip(n: number): Operator { + return { + factory: "skip", + body: (ctx) => { + const count = ctx.state.get() ?? 0; + if (count < n) { + ctx.state.set(count + 1); + return; // skipped → undirty RESOLVED + } + ctx.down([["DATA", ctx.depRecords[0].latest as S]]); + }, + }; +} + +/** + * takeWhile: emit values while `pred` holds; on the first value that fails `pred`, COMPLETE + * WITHOUT emitting it (RxJS default, non-inclusive). Terminal-is-forever once it completes. + */ +export function takeWhile(pred: (v: S) => boolean): Operator { + return { + factory: "takeWhile", + body: (ctx) => { + const v = ctx.depRecords[0].latest as S; + if (pred(v)) ctx.down([["DATA", v]]); + else ctx.down([["COMPLETE"]]); + }, + }; +} + +/** + * first: emit the first value matching `pred` (or simply the first value), then COMPLETE. EDGE + * (RxJS divergence, flagged — same class as last/find/elementAt): if the source COMPLETEs with no + * matching value, the substrate auto-cascades a bare COMPLETE (no value); RxJS `first()` throws + * EmptyError. Could align to `[[ERROR, EmptyError]]` (expressible) if strict RxJS parity is wanted. + */ +export function first(pred?: (v: S) => boolean): Operator { + return { + factory: "first", + body: (ctx) => { + const v = ctx.depRecords[0].latest as S; + if (!pred || pred(v)) ctx.down([["DATA", v], ["COMPLETE"]]); + // else skip → undirty RESOLVED (await the next matching value). + }, + }; +} + +/** + * last: emit the last value matching `pred` (or the last value) on the source's COMPLETE. + * `completeWhenDepsComplete:false + terminalAsRealInput:true`. EDGE (RxJS divergence, flagged): + * RxJS `last()` throws EmptyError when no value matched; we COMPLETE without a value (no throw) — + * the clean-slate substrate has no "complete-or-throw" terminal and SENTINEL forbids emitting a + * placeholder. + */ +export function last(pred?: (v: S) => boolean): Operator { + return { + factory: "last", + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const r = ctx.depRecords[0]; + if (r.batch) for (const v of r.batch) if (!pred || pred(v as S)) ctx.state.set({ v }); + if (r.terminal === true) { + const st = ctx.state.get<{ v: S }>(); + ctx.down(st ? [["DATA", st.v], ["COMPLETE"]] : [["COMPLETE"]]); + } + }, + }; +} + +/** + * find: emit the first value matching `pred`, then COMPLETE. EDGE (RxJS divergence, flagged): RxJS + * `find` emits `undefined` then COMPLETE when nothing matched — but `undefined` IS the SENTINEL + * (R-sentinel), so a not-found source COMPLETE here emits a bare COMPLETE (no value). + */ +export function find(pred: (v: S) => boolean): Operator { + return { + factory: "find", + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const r = ctx.depRecords[0]; + if (r.batch) { + for (const v of r.batch) { + if (pred(v as S)) { + ctx.down([["DATA", v], ["COMPLETE"]]); + return; + } + } + } + if (r.terminal === true) ctx.down([["COMPLETE"]]); // not found → bare COMPLETE + }, + }; +} + +/** + * elementAt: emit the value at zero-based `index`, then COMPLETE. EDGE (RxJS divergence, flagged): + * RxJS throws ArgumentOutOfRangeError if the source completes before `index`; we COMPLETE without + * a value (no throw), consistent with last/find. + */ +export function elementAt(index: number): Operator { + return { + factory: "elementAt", + opts: { completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const r = ctx.depRecords[0]; + let count = ctx.state.get() ?? 0; + if (r.batch) { + for (const v of r.batch) { + if (count === index) { + ctx.down([["DATA", v], ["COMPLETE"]]); + return; + } + count++; + } + ctx.state.set(count); + } + if (r.terminal === true) ctx.down([["COMPLETE"]]); // index out of range → bare COMPLETE + }, + }; +} + +/** Observer object for {@link tap} — lifecycle-aware side effects (RxJS tap observer form). */ +export interface TapObserver { + data?: (value: T) => void; + error?: (err: unknown) => void; + complete?: () => void; +} + +/** + * tap: invoke a side effect on each DATA (function form) or on data/error/complete (observer form); + * values pass through unchanged. The observer form reads source terminals + * (`completeWhenDepsComplete:false + errorWhenDepsError:false + terminalAsRealInput:true`) so it can + * call `error`/`complete` then forward the terminal; the function form lets the substrate + * auto-cascade terminals. + */ +export function tap(fnOrObserver: ((v: S) => void) | TapObserver): Operator { + if (typeof fnOrObserver === "function") { + const fn = fnOrObserver; + return { + factory: "tap", + body: (ctx) => { + const b = ctx.depRecords[0].batch; + if (b) + for (const v of b) { + fn(v as S); + ctx.down([["DATA", v]]); + } + }, + }; + } + const obs = fnOrObserver; + return { + factory: "tap", + opts: { + completeWhenDepsComplete: false, + errorWhenDepsError: false, + terminalAsRealInput: true, + }, + body: (ctx) => { + const r = ctx.depRecords[0]; + if (r.terminal === true) { + obs.complete?.(); + ctx.down([["COMPLETE"]]); + return; + } + if (r.terminal !== undefined) { + obs.error?.(r.terminal); + ctx.down([["ERROR", r.terminal]]); + return; + } + if (r.batch) + for (const v of r.batch) { + obs.data?.(v as S); + ctx.down([["DATA", v]]); + } + }, + }; +} + +/** + * onFirstData (alias `tapFirst`): invoke `fn` exactly once on the first qualifying value (default + * `where: v => v != null` — null/undefined pass through without counting as "first"), then pass all + * values through unchanged. The one-shot guard is per-node `ctx.state` (NOT a factory closure, so a + * second instantiation of the same factory re-arms — R-ctx-state). + */ +export function onFirstData( + fn: (v: S) => void, + opts?: { where?: (v: S) => boolean }, +): Operator { + const where = opts?.where ?? ((v: S) => v != null); + return { + factory: "onFirstData", + body: (ctx) => { + const b = ctx.depRecords[0].batch; + if (!b) return; + let fired = ctx.state.get() ?? false; + for (const v of b) { + if (!fired && where(v as S)) { + fired = true; + fn(v as S); + } + ctx.down([["DATA", v]]); + } + ctx.state.set(fired); + }, + }; +} + +/** tapFirst: alias of {@link onFirstData} (the one-shot companion to {@link tap}). */ +export const tapFirst = onFirstData; + +/** Options for {@link settle}. */ +export interface SettleOpts { + /** Consecutive no-change waves before declaring convergence + COMPLETE. */ + quietWaves: number; + /** Optional hard cap on total waves before forced COMPLETE. */ + maxWaves?: number; + /** Optional comparator; a DATA equal to the previous one does NOT reset the quiet counter. */ + equals?: (a: S, b: S) => boolean; +} + +/** Internal settle accumulator state (per-node, R-ctx-state). */ +interface SettleState { + last: S | undefined; + hasValue: boolean; + quiet: number; + waves: number; + done: boolean; +} + +/** + * settle: forward each DATA unchanged, watch for convergence, COMPLETE once the source has been + * quiet for `quietWaves` consecutive waves (or `maxWaves` elapsed). "Wave" = one fn invocation + * (a DATA batch OR a bare undirty RESOLVED end-of-wave). No polling (R-no-polling) — the counter + * advances only on real upstream waves. `settle`'s own `equals` is a body comparator, NOT the + * removed substrate `equals` (D49). + */ +export function settle(opts: SettleOpts): Operator { + const { quietWaves, maxWaves, equals } = opts; + if (!Number.isInteger(quietWaves) || quietWaves < 1) { + throw new RangeError(`settle: quietWaves must be a positive integer (got ${quietWaves})`); + } + if (maxWaves != null && (!Number.isInteger(maxWaves) || maxWaves < 1)) { + throw new RangeError(`settle: maxWaves must be a positive integer when set (got ${maxWaves})`); + } + return { + factory: "settle", + body: (ctx) => { + const st: SettleState = ctx.state.get>() ?? { + last: undefined, + hasValue: false, + quiet: 0, + waves: 0, + done: false, + }; + if (st.done) return; + st.waves++; + const b = ctx.depRecords[0].batch; + let sawChange = false; + if (b && b.length > 0) { + for (const v of b) { + const next = v as S; + const isChange = !st.hasValue || equals == null || !equals(st.last as S, next); + if (isChange) sawChange = true; + st.last = next; + st.hasValue = true; + ctx.down([["DATA", next]]); + } + } + st.quiet = sawChange ? 0 : st.quiet + 1; + const settled = st.hasValue && st.quiet >= quietWaves; + const exhausted = maxWaves != null && st.waves >= maxWaves; + if (settled || exhausted) { + st.done = true; + ctx.state.set(st); + ctx.down([["COMPLETE"]]); + return; + } + ctx.state.set(st); + // A no-DATA wave that didn't converge → bare undirty RESOLVED is synthesized by the + // substrate (D49); nothing to emit here. + }, + }; +} + +// ── Slice 3 — error-handling control (CSP-2.7, D40) ── +// rescue/catchError ABSORB the source ERROR (errorWhenDepsError:false) and read the error payload +// from `ctx.depRecords[0].terminal` (R-deps-terminal). They set completeWhenDepsComplete:false so a +// normal source COMPLETE is forwarded explicitly — which ALSO sidesteps B40 (the +// completeWhenDepsComplete:true × absorbed-errored-dep auto-complete gap): there is no auto-complete +// to block. valve is a [source, control] gate (state-verb-legitimate control input). + +/** + * rescue (alias `catchError`): replace an upstream ERROR with a recovered value. On source DATA → + * forward; on source ERROR → emit `recover(err)` as DATA (if `recover` throws, forward THAT as + * ERROR); on source COMPLETE → COMPLETE. After recovery the source is terminal-errored (dead) so no + * further values flow — the recovered value is the final cached value (matches the frozen pure-ts + * reference; it does NOT auto-COMPLETE, RxJS-`catchError(()=>of(x))` divergence, flagged). + */ +export function rescue(recover: (err: unknown) => S): Operator { + return { + factory: "rescue", + opts: { + errorWhenDepsError: false, + completeWhenDepsComplete: false, + terminalAsRealInput: true, + }, + body: (ctx) => { + const r = ctx.depRecords[0]; + if (r.batch) for (const v of r.batch) ctx.down([["DATA", v]]); + if (r.terminal === true) { + ctx.down([["COMPLETE"]]); + } else if (r.terminal !== undefined) { + // ERROR payload absorbed → recover. + try { + ctx.down([["DATA", recover(r.terminal)]]); + } catch (e) { + ctx.down([["ERROR", e]]); + } + } + }, + }; +} + +/** catchError: RxJS-named alias of {@link rescue}. */ +export const catchError = rescue; + +/** Options for {@link valve}. */ +export interface ValveOpts { + /** + * Optional AbortController (or a `() => AbortController | undefined` factory): on the control's + * truthy→falsy edge, `valve` calls `controller.abort()` — cancel an in-flight async boundary + * (LLM/fetch) the caller threaded `controller.signal` into. An external-resource cleanup (like + * `ctx.onDeactivation`), NOT an imperative reactive trigger (R-no-imperative). + */ + abortInFlight?: AbortController | (() => AbortController | undefined); +} + +/** + * valve: forward the source's DATA only while `control` (dep 1) is truthy; when closed, stay quiet + * (undirty RESOLVED). `[source, control]` declared deps + `partial:true` (the control wave can fire + * before the source ever delivers). `completeWhenDepsComplete:false` — closing the gate (control + * terminating) does NOT complete the valve; only the SOURCE's terminal (read via + * `terminalAsRealInput`) is forwarded. The `state`-verb control input is a legitimate external + * boundary (D4 / D54), not a forbidden imperative trigger. + */ +export function valve(opts?: ValveOpts): Operator { + const abortInFlight = opts?.abortInFlight; + return { + factory: "valve", + opts: { partial: true, completeWhenDepsComplete: false, terminalAsRealInput: true }, + body: (ctx) => { + const src = ctx.depRecords[0]; + const ctl = ctx.depRecords[1]; + const controlValue = ctl.latest as boolean | undefined; + + if (abortInFlight != null) { + // Fire abort on the truthy→falsy edge only (never on activation / no prior state). + const prev = ctx.state.get<{ ctl: boolean | undefined }>(); + if (prev?.ctl === true && !controlValue) { + const c = typeof abortInFlight === "function" ? abortInFlight() : abortInFlight; + c?.abort(); + } + ctx.state.set({ + ctl: controlValue === true ? true : controlValue == null ? undefined : false, + }); + } + + // Source terminal forwarding (control terminal is absorbed by completeWhenDepsComplete:false). + if (src.terminal === true) { + ctx.down([["COMPLETE"]]); + return; + } + if (src.terminal !== undefined) { + ctx.down([["ERROR", src.terminal]]); + return; + } + + if (!controlValue) return; // gate closed → quiet (undirty RESOLVED) + + const b = src.batch; + if (b && b.length > 0) { + for (const v of b) ctx.down([["DATA", v]]); + return; + } + // Gate just opened this wave (control fired, source didn't): re-emit the last source value. + if (ctl.batch && ctl.batch.length > 0 && src.prevData !== undefined) { + ctx.down([["DATA", src.prevData as S]]); + } + }, + }; +} diff --git a/packages/ts/src/graph/time.ts b/packages/ts/src/graph/time.ts new file mode 100644 index 00000000..765035e3 --- /dev/null +++ b/packages/ts/src/graph/time.ts @@ -0,0 +1,104 @@ +/** + * Wall-clock time operators (CSP-2.7 / D52 / B23). Authored as COMPOSITIONS over the already-active + * higher-order *Map machinery (D47) + the `timer` source — NO raw `setTimeout` in any operator body + * (R-no-raw-async / check-no-raw-async; `setTimeout` stays confined to `sources.ts`'s `timer`). The + * timer source's `onDeactivation` clearTimeout IS the reset/cancel: switchMap's removeDep on a new + * source value tears the in-flight timer down (the declarative equivalent of the frozen reference's + * imperative clearTimeout-then-setTimeout). Per-language (D6/D24), never in parity. + * + * Shipped this cut (clean *Map+timer): + * - delay = mergeMap(v → timer(ms)→v) (every value delayed ms, ALL kept, order preserved) + * - debounce(Time) = switchMap(v → timer(ms)→v) (cancel-and-restart; emit the latest after quiet ms) + * - throttle(Time) = exhaustMap(v → [v now, alive ms]) (leading edge: emit v, ignore for ms) + * + * Behavior on source COMPLETE (per-operator, NOT a uniform DROP): debounce/debounceTime EMIT their + * pending trailing value — switchMap does not cancel the in-flight inner on source COMPLETE (only a + * superseding source value cancels it via removeDep→onDeactivation→clearTimeout), so the timer still + * fires at `ms`, emits the debounced value, and only THEN does the operator COMPLETE (RxJS + * debounceTime parity). delay emits every still-pending delayed value then COMPLETEs (mergeMap keeps + * all inners). throttle/throttleTime have NO trailing value (leading-edge only, RxJS default + * trailing:false) — an open window inner is dropped when the operator completes. + * + * DEFERRED (flagged, NOT in this cut — each needs a mechanism beyond a clean *Map+timer projector): + * - audit/auditTime — trailing edge with a NON-resetting timer that emits the LATEST value seen in + * the window (not switchMap's cancel-restart, not exhaustMap's leading emit). + * - timeout — idle watchdog: ERROR if no DATA within ms; needs a resettable ERROR-timer + * raced against the source. + * - bufferTime/windowTime — buffer/window over interval(ms); bufferTime needs the interval wired as + * a self-added notifier dep, windowTime emits nested Node> (see the + * window-family deferral in combinators.ts). + */ + +import type { Node } from "../node/node.js"; +import { exhaustMap, mergeMap, switchMap } from "./higher-order.js"; +import { filter, initNode, map, merge, type Operator } from "./operators.js"; +import { of, timer } from "./sources.js"; + +/** A bare inner that emits `v` once after `ms`, then COMPLETEs (timer(ms)→map(v); auto-completes). */ +function delayedValue(v: S, ms: number): Node { + return initNode( + map(() => v), + [initNode(timer(ms), [])], + ); +} + +/** + * An inner that emits `v` IMMEDIATELY (on activation) and stays alive for `ms` (then COMPLETEs), + * emitting nothing more — the leading-edge throttle window. `merge(of(v), silentTimer)`: `of(v)` + * emits v + COMPLETE at t=0; `filter(()=>false)` over `timer(ms)` emits no DATA and COMPLETEs at + * t=ms; merge forwards v at t=0 and COMPLETEs (all deps complete) at t=ms. + */ +function throttleWindow(v: S, ms: number): Node { + const silentTimer = initNode( + filter(() => false), + [initNode(timer(ms), [])], + ) as Node; + return initNode(merge(), [initNode(of(v), []), silentTimer]); +} + +/** + * delay: shift every value by `ms`, preserving all values + order. `mergeMap` keeps every inner + * timer live (equal `ms`, so they fire in arrival order). + */ +export function delay(ms: number): Operator { + return { ...mergeMap((v) => delayedValue(v, ms)), factory: "delay" }; +} + +/** + * debounce (alias `debounceTime`): emit the latest value only after `ms` of quiet. `switchMap` + * cancels the prior timer (its onDeactivation clearTimeout) on each new source value and restarts — + * the declarative debounce. + * + * RxJS-7 divergence (reference rxjs@7.8, B44): RxJS `debounceTime` FLUSHES the pending value + * IMMEDIATELY when the source COMPLETEs (its `_complete` calls `debouncedNext()` before + * `complete()`). This composition instead emits the pending value when the in-flight `timer(ms)` + * fires — i.e. at `(last-value-time + ms)` — so if the source completes mid-window the trailing + * value arrives up to `ms` later than RxJS. Inherent to the `*Map`+`timer` form (the composition + * cannot flush early without detecting COMPLETE inside the switchMap inner); accepted, not a bug. + */ +export function debounce(ms: number): Operator { + return { ...switchMap((v) => delayedValue(v, ms)), factory: "debounce" }; +} + +/** debounceTime: RxJS-named alias of {@link debounce}. */ +export function debounceTime(ms: number): Operator { + return { ...switchMap((v) => delayedValue(v, ms)), factory: "debounceTime" }; +} + +/** + * throttle (alias `throttleTime`): leading-edge — emit a value immediately, then ignore the source + * for `ms`. `exhaustMap` drops new source values while the current window inner is alive; the + * window emits v at its leading edge and stays alive `ms`. + * + * Matches the RxJS-7 `throttleTime` DEFAULT (leading:true, trailing:false). The leading/trailing + * OPTIONS RxJS exposes are NOT provided (a capability gap, not a behavior divergence; B44) — add a + * trailing-window form if a consumer needs it. + */ +export function throttle(ms: number): Operator { + return { ...exhaustMap((v) => throttleWindow(v, ms)), factory: "throttle" }; +} + +/** throttleTime: RxJS-named alias of {@link throttle}. */ +export function throttleTime(ms: number): Operator { + return { ...exhaustMap((v) => throttleWindow(v, ms)), factory: "throttleTime" }; +} diff --git a/packages/ts/src/index.ts b/packages/ts/src/index.ts index 3e4a16df..918b1e29 100644 --- a/packages/ts/src/index.ts +++ b/packages/ts/src/index.ts @@ -16,6 +16,18 @@ export { type Pool, type PoolKind, } from "./dispatcher/index.js"; +export { + buffer, + bufferCount, + combine, + combineLatest, + concat, + race, + sample, + takeUntil, + withLatestFrom, + zip, +} from "./graph/combinators.js"; export type { DescribeEdge, DescribeNode, @@ -37,18 +49,37 @@ export { flatMap, mergeMap, type Project, + repeat, switchMap, } from "./graph/higher-order.js"; export type { NodeProfile, ObserveEvent, ObserveStream, Profile } from "./graph/inspect.js"; export { + catchError, distinctUntilChanged, + elementAt, filter, + find, + first, initNode, + last, map, merge, type Operator, + onFirstData, + pairwise, + reduce, + rescue, + type SettleOpts, scan, + settle, + skip, + type TapObserver, take, + takeWhile, + tap, + tapFirst, + type ValveOpts, + valve, } from "./graph/operators.js"; export { type AsyncSourceOpts, @@ -61,5 +92,12 @@ export { of, timer, } from "./graph/sources.js"; +export { + debounce, + debounceTime, + delay, + throttle, + throttleTime, +} from "./graph/time.js"; export { dynamicNode, Node, type NodeOptions, node, type Status } from "./node/node.js"; export * from "./protocol/messages.js"; diff --git a/packages/ts/src/node/node.ts b/packages/ts/src/node/node.ts index 2f1288dc..1b3c28c2 100644 --- a/packages/ts/src/node/node.ts +++ b/packages/ts/src/node/node.ts @@ -18,6 +18,7 @@ import { deferRewire, enterWave, exitWave } from "../batch/boundary.js"; import type { Ctx, CtxState, DepRecord, NodeFn, Sink } from "../ctx/types.js"; import { type Dispatcher, defaultDispatcher, type Handle } from "../dispatcher/index.js"; import { + isTerminal, isUpAllowed, type Message, messageTier, @@ -34,6 +35,14 @@ export type Status = | "completed" | "errored"; +/** + * SPIKE (design-review protocol-pull / pull:true) — NOT spec'd yet, pending /spec-amend (B). + * The well-known demand lock a `pull:true` node self-holds (quiet) and that a consumer RESUMEs + * to pull one delivery. A real impl would use a per-node lock for precise multi-pull routing; + * the shared symbol is sufficient to verify the wedge-fix premise (single pull node). + */ +export const PULL_DEMAND: unique symbol = Symbol("PULL_DEMAND"); + export interface NodeOptions { /** Pre-populate cache; source pushes [DATA, initial] on subscribe (R-initial). `null` is valid. */ initial?: T | null; @@ -51,6 +60,12 @@ export interface NodeOptions { resetOnTeardown?: boolean; /** PAUSE/RESUME behavior (R-pause-modes). Default true. */ pausable?: boolean | "resumeAll"; + /** + * SPIKE (protocol-pull): quiet-until-demanded source. Self-holds PULL_DEMAND (quiet): absorbs + * upstream DIRTY WITHOUT relaying it downstream (wedge fix) + no push-on-subscribe; a RESUME of + * PULL_DEMAND fires one delivery (pausable:true → latest; 'resumeAll' → backlog) then re-quiets. + */ + pull?: boolean; /** Buffer the last N outgoing DATA for late subscribers (R-replay-buffer). */ replayBuffer?: number; /** Mark this as a dynamicNode — fn gets ctx.track(i) for read-selection (R-dynamic-node / D35). */ @@ -89,6 +104,8 @@ export class Node { private readonly _resubscribable: boolean; private readonly _resetOnTeardown: boolean; private readonly _pausable: boolean | "resumeAll"; + /** SPIKE (protocol-pull / pull:true). */ + private readonly _pull: boolean; private readonly _replayN: number; private readonly _dynamic: boolean; readonly name?: string; @@ -163,6 +180,7 @@ export class Node { this._resubscribable = opts.resubscribable ?? false; this._resetOnTeardown = opts.resetOnTeardown ?? false; this._pausable = opts.pausable ?? true; + this._pull = opts.pull ?? false; this._replayN = opts.replayBuffer ?? 0; this._dynamic = opts.dynamic ?? false; this._pool = opts.pool ?? "sync"; @@ -195,6 +213,14 @@ export class Node { this._hasData = true; this._status = "settled"; } + + // SPIKE (protocol-pull): a pull node starts QUIET — self-hold the demand lock. + if (this._pull) this._pauseLockset.add(PULL_DEMAND); + } + + /** SPIKE (protocol-pull): true while a pull node is quiet (holds its own demand lock). */ + private _isPullQuiet(): boolean { + return this._pull && this._pauseLockset.has(PULL_DEMAND); } get cache(): T | undefined { @@ -247,9 +273,12 @@ export class Node { if (this._replayN > 0 && this._replayRing.length > 0) { // R-replay-buffer: late subscriber gets the last N DATA after START. for (const v of this._replayRing) sink(["DATA", v]); - } else if (this._hasData) { + } else if (this._hasData && !this._isPullQuiet()) { + // SPIKE (protocol-pull): a quiet pull node does NOT push its cached value on + // subscribe — it stays silent until demanded (else every new subscriber would + // trigger a materialization). START only. sink(["DATA", this._cache]); - } else if (this._status === "dirty") { + } else if (this._status === "dirty" && !this._isPullQuiet()) { sink(["DIRTY"]); } @@ -623,33 +652,33 @@ export class Node { return; } - if (t === "COMPLETE") { - this._depTerminal[idx] = true; - // R-terminal-settles-dirty (B35): a terminal RELEASES this dep's outstanding in-wave - // DIRTY contribution (the exactly-one-settle invariant) — exactly as DATA/RESOLVED/ - // INVALIDATE do (a dirty-then-terminal-without-DATA dep would otherwise strand _pending - // and wedge the node, the deadlock R-invalidate-idempotent prevents for INVALIDATE). + if (isTerminal(t)) { + // Tier 5 (R-tier / D34): COMPLETE | ERROR — ONE branch routed by the CENTRAL tier table, + // not a per-variant string check (feedback_use_tier_for_signal_routing). The shared terminal + // bookkeeping (record the terminal + release the in-wave DIRTY) runs for ANY tier-5 message; + // only the COMPLETE-vs-ERROR cascade differs, so discriminate by the type within the tier. + const isError = t === "ERROR"; + const errPayload = isError ? (msg as readonly ["ERROR", unknown])[1] : undefined; + // Record this dep's terminal: the ERROR payload, or `true` for COMPLETE. + this._depTerminal[idx] = isError ? errPayload : true; + // R-terminal-settles-dirty (B35): a terminal RELEASES this dep's outstanding in-wave DIRTY + // contribution (the exactly-one-settle invariant) — exactly as DATA/RESOLVED/INVALIDATE do + // (a dirty-then-terminal-without-DATA dep would otherwise strand _pending and wedge the node, + // the deadlock R-invalidate-idempotent prevents for INVALIDATE). this._releaseDepDirty(idx); - if (this._completeWhenDepsComplete && this._allDepsComplete()) { - this._down([["COMPLETE"]]); // auto-cascade → node itself terminal (_pending moot) - } else if (this._terminalAsRealInput) { - this._maybeRun(); // rescue/reduce/*Map: the fn reads ctx.depRecords[idx].terminal - } else { - // absorbed terminal, NOT an input: the dep's signalled change did not materialise - // (no DATA) → un-dirty downstream, keep cache (R-resolved-undirty balance). - this._settleAfterAbsorbedTerminal(); - } - return; - } - - if (t === "ERROR") { - this._depTerminal[idx] = msg[1]; - this._releaseDepDirty(idx); // R-terminal-settles-dirty (B35), as COMPLETE above - if (this._errorWhenDepsError) { - this._down([["ERROR", msg[1]]]); // auto-cascade → node itself terminal + if (isError && this._errorWhenDepsError) { + this._down([["ERROR", errPayload]]); // auto-cascade ERROR → node itself terminal } else if (this._terminalAsRealInput) { - this._maybeRun(); // rescue/catch: the fn reads the dep's terminal (the error) + this._maybeRun(); // rescue/reduce/catch/*Map: the fn reads ctx.depRecords[idx].terminal + } else if (this._completeWhenDepsComplete && this._allDepsTerminal()) { + // R-deps-terminal auto-COMPLETE + B42: COMPLETE once ALL deps are TERMINAL (each COMPLETE + // or an absorbed ERROR) — so an absorbed-error dep terminating LAST still fires the + // cascade. terminalAsRealInput is checked FIRST so a rescue recovers via _maybeRun rather + // than being preempted (no operator sets both completeWhenDepsComplete:true + tari:true). + this._down([["COMPLETE"]]); } else { + // absorbed terminal, NOT an input + not auto-completing: the dep's signalled change did + // not materialise (no DATA) → un-dirty downstream, keep cache (R-resolved-undirty balance). this._settleAfterAbsorbedTerminal(); } return; @@ -746,6 +775,12 @@ export class Node { private _markDirty(): void { this._status = "dirty"; + // SPIKE (protocol-pull): while quiet, ABSORB the upstream DIRTY — do NOT relay it downstream. + // This is the P0b wedge fix: a quiet pull node that relayed DIRTY but withheld the settle + // (coalesced by pause) wedged every downstream's two-phase _pending. The downstream learns of + // changes via the push STREAM port, not the silent snapshot port; on demand the pull node + // emits a fresh wave. Internal dep dirty-accounting (the DIRTY-branch _pending++) is untouched. + if (this._isPullQuiet()) return; if (!this._emittedDirtyThisWave) { this._emittedDirtyThisWave = true; this._emitToSubs(["DIRTY"]); @@ -1196,6 +1231,11 @@ export class Node { this._pausedDepWaveOccurred = false; this._tryRun(); } + // SPIKE (protocol-pull): one demand = one delivery — re-quiet after the pulse so the next + // upstream change is again absorbed silently (1:1, sub-decision #2). The fire above already + // emitted (true → latest via _tryRun; 'resumeAll' → backlog via the buffer drain) while NOT + // quiet, so downstream got the delivery; re-acquiring the lock now returns to silence. + if (this._pull) this._pauseLockset.add(PULL_DEMAND); } /** Should an outgoing settle slice be deferred into the pause buffer? */ @@ -1224,9 +1264,14 @@ export class Node { this._emitToSubs(["INVALIDATE"]); } - private _allDepsComplete(): boolean { + // B42 (R-deps-terminal): ALL deps TERMINAL = every dep has reached COMPLETE *or* an ABSORBED + // ERROR. Block only on a LIVE dep (_depTerminal[i] === undefined); an errored dep (terminal = the + // error payload, which R-data-payload guarantees is !== undefined) COUNTS as terminal-done. Was + // `tm !== true`, which wedged a node whose errorWhenDepsError:false dep ERRORed (it never + // auto-completed even after every other dep completed). Drives the completeWhenDepsComplete cascade. + private _allDepsTerminal(): boolean { if (this._deps.length === 0) return false; - for (const tm of this._depTerminal) if (tm !== true) return false; + for (const tm of this._depTerminal) if (tm === undefined) return false; return true; } diff --git a/packages/ts/src/protocol/messages.ts b/packages/ts/src/protocol/messages.ts index 313ba5ff..d5ad4215 100644 --- a/packages/ts/src/protocol/messages.ts +++ b/packages/ts/src/protocol/messages.ts @@ -62,6 +62,16 @@ export function isDeferredTier(t: MessageType): boolean { return TIER[t] >= 3; } +/** + * A TERMINAL message = tier 5 (COMPLETE | ERROR), R-tier / D34. Detected via the CENTRAL tier + * table, NOT a per-variant `=== "COMPLETE" || === "ERROR"` check — so terminal routing stays + * driven by the one const table (feedback_use_tier_for_signal_routing); discriminate COMPLETE vs + * ERROR within the tier by the message type only where the handling actually differs. + */ +export function isTerminal(t: MessageType): boolean { + return TIER[t] === 5; +} + /** * ctx.up carries control tiers only (R-ctx-up / DR-5): DIRTY, PAUSE, RESUME, * INVALIDATE, TEARDOWN. DATA/RESOLVED (tier 3) and COMPLETE/ERROR (tier 5) are From f366a5bde8148f3ec978507e1c12aa9e43a1dbb5 Mon Sep 17 00:00:00 2001 From: David Chen Date: Sun, 31 May 2026 10:25:44 -0700 Subject: [PATCH 012/175] feat(ts): add agent context and skills documentation for GraphReFly - Introduced a new `AGENTS.md` file detailing the TypeScript implementation of GraphReFly, outlining the authority, workflow rules, and commands for developers. - Created multiple SKILL documents for various functionalities, including `conformance`, `dashboard`, `decision-guard`, `design-review`, `dev-dispatch`, `graph-animation`, and `qa`, each providing specific guidance on their respective workflows and usage. - Enhanced documentation clarity and structure to support the clean-slate redesign, ensuring that users can easily navigate and understand the new processes and skills available in the GraphReFly ecosystem. - Established a comprehensive framework for maintaining consistency and adherence to design principles across different language implementations. --- .agents/skills/conformance/SKILL.md | 61 +++ .agents/skills/dashboard/SKILL.md | 33 ++ .agents/skills/decision-guard/SKILL.md | 87 ++++ .agents/skills/design-review/SKILL.md | 146 ++++++ .agents/skills/dev-dispatch/SKILL.md | 123 ++++++ .agents/skills/graph-animation/SKILL.md | 418 ++++++++++++++++++ .agents/skills/qa/SKILL.md | 110 +++++ .agents/skills/research/SKILL.md | 143 ++++++ .agents/skills/spec-amend/SKILL.md | 34 ++ AGENTS.md | 82 ++++ .../docs/SESSION-reactive-linear-and-git.md | 396 +++++++++++++++++ archive/docs/design-archive-index.jsonl | 1 + packages/ts/src/__tests__/conformance.test.ts | 152 +++++++ packages/ts/src/__tests__/pull.test.ts | 163 +++++++ packages/ts/src/ctx/types.ts | 20 +- packages/ts/src/node/node.ts | 269 +++++++++-- 16 files changed, 2197 insertions(+), 41 deletions(-) create mode 100644 .agents/skills/conformance/SKILL.md create mode 100644 .agents/skills/dashboard/SKILL.md create mode 100644 .agents/skills/decision-guard/SKILL.md create mode 100644 .agents/skills/design-review/SKILL.md create mode 100644 .agents/skills/dev-dispatch/SKILL.md create mode 100644 .agents/skills/graph-animation/SKILL.md create mode 100644 .agents/skills/qa/SKILL.md create mode 100644 .agents/skills/research/SKILL.md create mode 100644 .agents/skills/spec-amend/SKILL.md create mode 100644 AGENTS.md create mode 100644 archive/docs/SESSION-reactive-linear-and-git.md create mode 100644 packages/ts/src/__tests__/pull.test.ts diff --git a/.agents/skills/conformance/SKILL.md b/.agents/skills/conformance/SKILL.md new file mode 100644 index 00000000..2acd3479 --- /dev/null +++ b/.agents/skills/conformance/SKILL.md @@ -0,0 +1,61 @@ +--- +name: conformance +description: "Behavioral conformance check across GraphReFly language runtimes (ts/rust/py) for the clean-slate redesign. Replaces the old structural 'parity' diff. Parity = does each runtime satisfy the wave-protocol behavior (conformance scenarios) + dispatcher contract — NOT 'do the symbol sets match'. Use after implementing/changing substrate behavior in any runtime, or when adding a new protocol rule. Authors/runs language-agnostic scenarios and updates conformance.jsonl runtime status. Triggers: 'conformance', 'cross-lang check', 'does rust match', 'parity', 'run the conformance suite', 'is the substrate behavior consistent'." +argument-hint: "[rule-id | scenario-id | 'full'] [optional: runtime ts|rust|py]" +--- + +You are executing **conformance** for the clean-slate GraphReFly redesign. + +**Parity is behavioral, not structural (D24).** There is NO `Impl` symbol-set to diff and NO +cross-track-ledger. Operators / sugar / inspection are **per-language and never in parity** +(D6/D27 — graph-layer wraps everything to `(ctx)=>void` before register). The ONLY parity +surface is: **wave-protocol behavior + dispatcher contract + handle format**. Conformance = +each runtime passes the same language-agnostic scenarios. + +## Authority + +| Source | Role | +|---|---| +| `~/src/graphrefly/spec/conformance.jsonl` | The scenario registry: `{id, name, covers:[rule-id], runtimes:{ts,rust,py}, status, note}`. | +| `~/src/graphrefly/spec/rules.jsonl` | The rules scenarios pin (`covers` must resolve here). | +| `~/src/graphrefly/spec/protocol.proto` | Protocol-contract IDL (DR-2) — the light structural anchor codegen'd into each runtime's interface stub. | +| `~/src/graphrefly/formal/*.tla` | TLA+ model (γ); property tests mirror its invariants. | + +## Scope from $ARGUMENTS + +- **rule-id** (e.g. `R-diamond`) → all scenarios whose `covers` includes it. +- **scenario-id** (e.g. `C-1`) → that scenario. +- **full** → every `status:"required"` scenario. +- optional **runtime** → restrict to one arm. + +## Phase 1 — scenario integrity + +1. Load `conformance.jsonl` + `rules.jsonl`. Verify every `covers` rule-id resolves (else HALT — fix the scenario or add the rule via `/spec-amend`). +2. List the **DR-5 required hard scenarios** and their status: `C-1` cross-graph diamond, `C-2` async-result-at-paused-node, `C-3` INVALIDATE×ctx.state×onInvalidate, `C-4` mixed sync/async diamond, `C-5` PAUSE-lockset multi-source. These are the load-bearing ones — behavioral parity is a blank cheque until they're green on each shipped runtime. + +## Phase 2 — run / verify per runtime + +For each in-scope `(scenario, runtime)`: +1. Locate/author the scenario harness in that runtime's conformance test dir (language-agnostic spec → per-runtime adapter; the scenario describes observable wave behavior, not a symbol call). +2. Run it. Record outcome. +3. Update `conformance.jsonl` `runtimes.`: `"todo" | "poc-pass" | "pass" | "fail"`. +4. Mirror the invariant as a property test (fast-check ts ↔ proptest rust ↔ hypothesis py) where the rule is property-shaped (L5-Q2 / D14). + +## Phase 3 — report + +| scenario | covers | ts | rust | py | verdict | +|---|---|---|---|---|---| + +- **Behavior drift** = same scenario, different observable outcome across runtimes → this is the ONLY kind of parity gap. File it as a substrate bug in the lagging runtime (route fix via `/dev-dispatch` on that package). +- **Missing scenario** for a rule (rule's `covers_by` empty) → author it (this is the real risk under behavioral parity: untested behavior can drift silently — D24 residual). Flag via `/dashboard` Gaps (uncoveredRules). +- **NOT a gap:** a runtime having a different operator set / different sugar / different inspection ergonomics. Those are per-language by design — do not report them. + +## Phase 4 — gate + +Run `node ~/src/graphrefly/dashboard/build.mjs --check` (scenario↔rule links intact). For a runtime +to be declared "conformant", all `status:"required"` scenarios must be `"pass"` on its arm. + +## Boundaries + +Does NOT diff symbol sets (that's the retired structural model). Does NOT touch operators/sugar/inspection +(per-language). New protocol behavior must go through `/spec-amend` FIRST (scenario authored before code). diff --git a/.agents/skills/dashboard/SKILL.md b/.agents/skills/dashboard/SKILL.md new file mode 100644 index 00000000..a2903ef4 --- /dev/null +++ b/.agents/skills/dashboard/SKILL.md @@ -0,0 +1,33 @@ +--- +name: dashboard +description: "Build / check the GraphReFly internal docs dashboard (jsonl single-source -> generated HTML with progress, structure map, gaps, search). Use when the user wants to see global project state, regenerate the dashboard, run the docs consistency gate (broken links / orphans / coverage gaps), or after editing any jsonl in ~/src/graphrefly (decisions/plan/spec/sessions/guide). Triggers: 'build the dashboard', 'check docs consistency', 'what are the gaps', 'show progress', 'regenerate dashboard', 'doc gate'." +argument-hint: "[--check (gate only) | (default: build + report)]" +--- + +You are executing **dashboard** for the clean-slate GraphReFly redesign. + +**Repo:** `~/src/graphrefly` (clean-slate branch). All structured docs are jsonl (single source of truth, decision 2); the dashboard renders them into one searchable HTML view for the maintainer (decision 3). Schema contract: `~/src/graphrefly/dashboard/README.md`. + +## What this skill does + +1. **Run the generator:** + - `node ~/src/graphrefly/dashboard/build.mjs` → writes `dashboard/dashboard.html` + prints counts / gaps / broken-links / orphans report. + - `node ~/src/graphrefly/dashboard/build.mjs --check` → consistency gate only; **non-zero exit on broken links** (use as a pre-commit / CI gate). +2. **Interpret the report** for the user: + - **counts** — per-jsonl row counts (decisions/phases/rules/conformance/...). + - **gaps** — designPhases (status=design|gap) · openDecisions · deferredBacklog · uncoveredRules (no conformance) · todoConformance (runtimes=todo). This answers "哪里还有缺口". + - **broken links** — must be zero (session.locks → decision; phase.sessions → session; conformance.covers → rule; flowchart.explains → rule|D#). Legacy 3-digit D### / R# refs are external (old main), reported separately as OK. + - **orphans** — decisions referenced by no session (informational). +3. **If broken links exist:** locate the offending jsonl row, fix the id reference (or add the missing record), re-run `--check`. + +## When the jsonl changed + +After any edit to `decisions/`, `plan/`, `spec/`, `sessions/`, `guide/` jsonl — run `--check` to catch dangling references immediately (fixes P4 stale-premise + P6 link-rot). The generator is the enforcement mechanism for "single canonical, no broken cross-refs." + +## UI styling + +`build.mjs` emits a **placeholder shell** with the data model embedded. Visual design / interactive search is a separate `/frontend-design` pass — do NOT hand-style the HTML here; keep `build.mjs` focused on the data model + consistency checks. The dogfood endgame (phase CSP-8) rebuilds this dashboard *with GraphReFly itself* (jsonl producer → reactive views → HTML effect). + +## Output + +The counts + gaps + link-health report, a plain-language "where are the gaps / what's the progress" summary, and (unless `--check`) confirmation that `dashboard.html` was written. Flag any broken link as a blocker to fix before commit. diff --git a/.agents/skills/decision-guard/SKILL.md b/.agents/skills/decision-guard/SKILL.md new file mode 100644 index 00000000..78f64884 --- /dev/null +++ b/.agents/skills/decision-guard/SKILL.md @@ -0,0 +1,87 @@ +--- +name: decision-guard +description: "GraphReFly clean-slate decision-consistency check. Loads the user's locked values/principles + the unified D-numbered decision log (decisions.jsonl) + recurring decision-process patterns. Use BEFORE answering any question of the form 'is this consistent with our decisions?', 'should I pick option A/B/C?', 'what about this proposed fix?', 'is X part of our scope?', 'is this a regression on a prior decision?'. Triggers: 'decision check', 'drift check', 'align check', 'is this consistent', 'should I pick', 'what about this', 'is this in scope', 'consistency review'." +argument-hint: "[short context of what you're being asked about — paste the proposal if relevant]" +--- + +# decision-guard — recall and apply locked decisions, values, invariants (clean-slate) + +**Purpose.** Conversations lose context-window state quickly. This is the canonical recall +surface for the **clean-slate** redesign: invoke BEFORE answering decision questions to anchor +against the user's locked positions and prevent silent drift — especially when a chat proposes +a scope expansion mid-implementation, presents A/B/C as a fork, uses "completeness" to justify +expanding a locked slice, or builds on a premise that may be stale. + +> **Clean-slate retired the old port model.** Do NOT reach for `rust-port-decisions.md`, +> `cross-track-ledger.md`, the `Impl` parity contract, `BindingBoundary`, the actor model, or +> 3-digit D### port decisions — those are old-`main` history. The clean-slate decision +> authority is below. + +## Sources (load in order) + +| Source | Role | +|---|---| +| `~/src/graphrefly/decisions/decisions.jsonl` | **Unified D# log** (D1–D33 + DR-*). Canonical record: `{id, layer, question, decision, rationale, supersedes, status}`. | +| `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) | Full design narrative + 8 forced constraints + spec-amendment list + conformance hard scenarios. | +| `~/src/graphrefly/plan/antipatterns.jsonl` | Lessons / anti-patterns to flag against. | +| `~/src/graphrefly/spec/rules.jsonl` | Protocol rules — for "does the spec already pin this?". | +| Memory `feedback_*` files | The user's durable values/principles (below). | + +## Locked values (durable — cite by name) + +1. **No backward compat** (pre-1.0): structurally cleaner option, no legacy shims. `feedback_no_backward_compat`. +2. **No imperative triggers** in public API: reactive `ctx.up`/signals, not emitters/callbacks/timers+set; remove imperative paths when no caller depends. `feedback_no_imperative`. +3. **Single source of truth**: one canonical per concern; index points, never duplicates. `feedback_single_source_of_truth`. +4. **No autonomous decisions** (hard rule): surface spec↔code conflicts; don't silently pick; file-by-file review for multi-file rewrites. `feedback_no_autonomous_decisions`. +5. **No implement without approval**: decisions locked ≠ implementation approved. `feedback_no_implement_without_approval`. +6. **Verify premise before greenfield**: design tables lag code — grep symbols + check landed markers before a 9Q; stale premise = HALT. `feedback_verify_premise_before_greenfield`. +7. **Latest versions + context7** for current API docs. `feedback_latest_versions_context7`. +8. **Long-command observation discipline** (run-logged + DONE sentinel; no tail; no sleep-poll) and **subagent bg hygiene** (sync-verify or teardown before return). `feedback_long_command_observation`, `feedback_subagent_bg_hygiene`. + +## Clean-slate floor (never violate) + +**Sacred (L0.7):** topology declarative/serializable/inspectable · wave protocol is a public spec · +wave protocol impl is **sync** · all fn go through dispatcher. + +**Forced (F-*):** F-PERF (budget every abstraction) · F-PROTO-SPEC (spec+TLA++property) · +F-SYNC-CORE (dispatcher.invoke sync void) · F-DISPATCH-ALL (no inline-fn bypass) · +F-GRAPH-FIRST-API · F-NO-WEDGE-CUT (every primitive ≥2 segments) · +F-NO-IMPL-DEFINED (spec-locked or explicitly undefined-by-design) · F-NO-LLM-ONLY. + +**Red flags (HALT if proposed):** async in the sync wave core (async lives only in pools/wire-bridge) · +inline-fn bypassing dispatcher · a primitive serving only LLM workflows · a protocol behavior left +"implementation-defined" · user-replaceable onMessage/onSubscribe · adding a 10th tier casually · +graph-level shared mutable state accessed implicitly (must be explicit node + dep). + +## Decision-process patterns (apply in order) + +1. **Identify the governing D#.** Grep `decisions.jsonl` by `layer`/keyword. Is the proposal within a locked D's scope? Mid-implementation scope expansion = anti-pattern unless promoted to a NEW D#. +2. **Check the spec.** Does `rules.jsonl` pin the behavior? If yes, follow it — divergence is a bug, not a design call. If silent/ambiguous → real design HALT. +3. **Verify premise (value 6).** Has the symbol/surface already landed? grep before designing new surface. +4. **Apply values + floor.** Especially: no autonomous decisions, no imperative, single source of truth, sync-core/async-at-boundary, F-* constraints. +5. **Verdict:** `consistent (cite D#)` / `regression on D#` / `out-of-scope` / `needs new decision (don't auto-pick)`. +6. **Routing:** + - New fork, no governing D# → present options + recommend, **do NOT lock** → that's `/design-review` → user approval → append `decisions.jsonl`. + - Changes protocol behavior → `/spec-amend` (spec-first). + - Cross-runtime parity concern → `/conformance` (behavioral scenario, not structural diff). + +## Common decision shapes + +- **A/B/C (fix shape):** default = the option that **structurally extends an existing pattern** beats per-site workarounds. +- **Completeness vs discipline:** if a proposal closes a real semantic gap → formalize as a NEW D# (don't continue under the original D's banner). If speculative scope expansion → revert. +- **Orthogonal sub-decisions:** before locking two "orthogonal" sub-decisions, sketch ONE input exercising BOTH — confirm orthogonality survives the example (antipatterns.jsonl; the 30-second check that catches coupling at design-time). + +## Scope boundaries + +Read-mostly. Loads decisions + values + patterns; produces **decision + reasoning + relay-ready text**. +Does NOT run gates, apply fixes, or author scenarios — those are `/dev-dispatch` / `/qa` / `/conformance` +after a decision locks. Invoke when the question is "what should I decide?", not "what should I do?". + +## Update protocol + +When a new D# locks (after user approval): append to `~/src/graphrefly/decisions/decisions.jsonl` +(`{id, layer, date, question, decision, rationale, supersedes, status:"locked", session}`), update the +session's `locks` in `sessions.jsonl`, and run `node ~/src/graphrefly/dashboard/build.mjs --check`. +When a new anti-pattern recurs: append to `~/src/graphrefly/plan/antipatterns.jsonl` (+ a `feedback_*` +memory if generalizable). When a new durable value surfaces: add a `feedback_.md` memory + a +pointer line here. diff --git a/.agents/skills/design-review/SKILL.md b/.agents/skills/design-review/SKILL.md new file mode 100644 index 00000000..5bc4267f --- /dev/null +++ b/.agents/skills/design-review/SKILL.md @@ -0,0 +1,146 @@ +--- +name: design-review +description: "Validate the design of a new primitive or API surface against the 5-question lens (Q5–Q9 from the per-unit review format). Use BEFORE coding (or right after a sketch lands) when adding a new public API / pattern factory / domain primitive. Triggers: 'design review', 'review the design', 'is this the right shape', 'before I implement'. Different from /qa — that finds bugs in landed code; this validates abstraction + long-term shape + reactive composability + alternatives." +argument-hint: "[ | | --diff] [optional context]" +--- + +You are executing the **design-review** workflow for the **clean-slate GraphReFly** redesign. + +This skill applies the 5 design-review questions (Q5–Q9 of the 9-question per-unit format) to a **single-symbol / single-file / single-diff** review. Use it BEFORE coding a new public API / sugar factory / operator / inspection surface — or right after a sketch lands, before tests. Different from `/qa` (finds bugs in landed code) and `/decision-guard` (recalls locked decisions); this validates abstraction + long-term shape + reactive composability + alternatives, and its output may become a new `D#` (architectural lock → user approval → append `decisions.jsonl`). + +Clean-slate code lives in **`packages/ts/src/`** (`@graphrefly/ts`, D32). The language-neutral authority is **`~/src/graphrefly`** jsonl (branch `clean-slate`) — when this skill and that repo disagree, that repo wins (AGENTS.md). + +> **Stale-infra guard.** Do NOT cite the retired port-model: `packages/pure-ts/**` (frozen read-only reference only, D41), `docs/implementation-plan.md` / `optimizations.md` / `roadmap.md` / `test-guidance.md` / `docs-guidance.md`, `GRAPHREFLY-SPEC.md` / `COMPOSITION-GUIDE.md` (migrated to `spec/rules.jsonl` + `guide/guide.jsonl`, B7), `describe({format})` (D39: renderers are pure fns over the snapshot, NOT a `format` option), the `Impl`/facade/actor model. + +**When to use:** before a new public symbol in `packages/ts/src/graph/` (sugar / operator / inspection) or a new substrate primitive in `packages/ts/src/{node,dispatcher,ctx,protocol,batch}/`; right after sketching a factory; when two implementations exist and you need a principled pick; when `/dev-dispatch` Phase 2 needs to go deeper than its default template. +**When NOT:** bug fixes / pure refactors → `/qa`; already-approved work → proceed; trivial additive changes (a JSDoc field, a docstring). + +User context: $ARGUMENTS + +--- + +## Phase 0: Scope resolution + +Resolve the target(s) from `$ARGUMENTS`: + +1. **`--diff` / no args** — review the new public symbols in the uncommitted diff. Enumerate via `git diff --name-only HEAD` + `git status --short`, filtered to `packages/ts/src/**` files that introduce new exports. +2. **``** — public symbols in that file. +3. **``** — Grep-locate, then review. +4. **Multiple targets** — apply Q5–Q9 to each, then add Phase 2 synthesis. + +Read in parallel before reviewing (clean-slate authority): + +- `~/src/graphrefly/spec/rules.jsonl` — the R-* rules your target touches (the design invariants). +- `~/src/graphrefly/decisions/decisions.jsonl` — the governing D# (or `/decision-guard` to recall the floor/values). +- `~/src/graphrefly/plan/phases.jsonl` — the CSP-* phase the target belongs to (locked vs open-design); if the target isn't sequenced yet, the review's output may need a new phase / backlog entry. +- `~/src/graphrefly/guide/guide.jsonl` (G-composition) — composition patterns (lazy activation, subscription order, SENTINEL/prevData guards, feedback cycles). +- `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) — the F-* constraints + the why behind locks. +- The target file(s) + 1–2 closest existing primitives in the same dir (precedent). +- **Frozen reference (D41):** `packages/pure-ts/**` + `~/src/callbag-recharge` for analogous prior art (operator behavior, edge cases, test structure) — NOT the authority. + +--- + +## Phase 1: Per-target review (Q5–Q9) + +For each target, produce a structured report covering all five questions. Be specific, quote `file:line`. Cap each answer ~150 words. + +### Q5 — Is this the right abstraction? Could it be more generic? + +- **Layer placement.** Substrate (`node`/`dispatcher`/`ctx`/`protocol`/`batch` — a protocol primitive, must stay thin per R-node-thin) vs graph-layer (`graph/` — sugar / operator / inspection, per-language per D6/D24)? Mismatched layer is the most common drift signal. A new **verb** is a constitutional change (8-verb closed set, D4) — almost certainly the target is sugar, not a verb. +- **Decomposition.** Could the body split into 2+ smaller primitives that compose? (F-NO-WEDGE-CUT: each must serve ≥2 segments.) +- **Generalization.** A 2+ similar primitive nearby hinting at a shared abstraction? Could `T = number` be `T = unknown` without losing safety? +- **Naming.** Does the name describe what it **returns/produces** (composable) or what it does internally (rots)? D6 real factory names show in `describe`. + +> **Layer / Decomposition / Generalization / Naming:** … + +### Q6 — Right long-term solution? Caveats? Maintenance burden? + +- **6-month lens.** What forces this to evolve — spec/conformance changes, rust/py arm parity (D24), F-PERF budget? +- **Hidden invariants** the type system can't express — list each as `INVARIANT: …` (subscribe order before kick; first-run gate; SENTINEL = `prevData === undefined`; sync-vs-async strategy; stable refs; equals identity). +- **Constraint locks** (positional args can't grow options; hardcoded enum can't extend). +- **Doc debt** (contract knowable from JSDoc, or only from the body?). + +> **6-month risk / Hidden invariants / Constraint locks / Doc debt:** … + +### Q7 — Can we simplify it? Reactive, composable, explainable? + +The **explainability check** — a primitive's reactive shape is only as good as its `describe()` snapshot (D39). + +- **Wire a minimal composition** (≥2 sources → target → ≥1 sink). Predict `describe()` — a flat JSON-serializable snapshot; renderers (pretty/mermaid/d2) are pure fns over it (D39), NOT a `describe({format})` option. If you can't predict the output, the topology is too imperative. +- **Island check.** A node with zero in-edges AND zero out-edges (not an entry/exit) is a smell. +- **Imperative escape paths.** Search for: emit/set/callback wiring that bypasses the graph (R-no-imperative); `.cache` reads inside reactive fn bodies (R-data-not-peek — data moves via messages); raw `Promise`/`setTimeout`/`queueMicrotask` outside a source/pool (R-no-raw-async / F-SYNC-CORE); hardcoded `type === "DATA"` instead of `messageTier` (R-tier); an inline fn bypassing the dispatcher (R-dispatch-all). +- **SENTINEL / prevData guards.** Never-emitted detection via `ctx.prevData[i] === undefined` (the canonical detector); fix eager-placeholder upstreams rather than bolting on companions. +- **Feedback cycles.** A fn that re-drives its own dep mid-wave is a wave-level ERROR (D37/R-reentrancy), not iteration; legit accumulation = `ctx.state` (scan), not a topological cycle. + +> **Topology / Imperative leaks / cache reads / Feedback cycles / Simplifications:** … + +### Q8 — Alternative implementations (A / B / C) + +Sketch **≥2** named alternatives. For each: **Shape** (1–3 line pseudo-sig), **Pros** (2–4), **Cons** (2–4), **Precedent** — does the shape exist in the frozen `packages/pure-ts/**` reference (D41), `callbag-recharge`, or RxJS? Cite if so. Don't pick a winner yet — that's Q9. + +> **A. {name}** — sketch / Pros / Cons / Precedent +> **B. {name}** — … + +### Q9 — Recommendation + coverage check + +Pick the recommended alternative; build a coverage matrix (each Q5–Q8 concern → recommended alt covers it? yes / partially / no — because …). For any `partially`/`no`, name the residual risk: accept it (justify), add a `backlog.jsonl` follow-up, or pick a different alternative. End with: + +> **Recommendation:** {alt}, because {2–3 reasons grounded in Q5–Q8}. +> **Residual risks:** {none, OR 1–2}. +> **Implementation guidance:** {next step — usually a draft `D#` for approval, or a sub-decision the user must answer first}. + +If the design is an architectural lock, draft the `D#` (`{id, layer, date, question, decision, rationale, supersedes, status}`) for the user to approve BEFORE append — do NOT auto-lock (no-autonomous-decisions). A wave-protocol behavior change routes to `/spec-amend` (spec-first), not a direct edit. + +--- + +## Phase 2: Cross-cutting synthesis (multi-target only) + +Apply only when reviewing multiple targets in one pass: + +- **Naming consistency** (`extract` vs `select` vs `pick` for the same role = drift). +- **Argument-shape consistency** (options-bag vs positional applied the same way). +- **Composition direction** (do the targets' input/output shapes line up to compose?). +- **Repeated patterns** (the same SENTINEL-gate / subscribe-order / batch-on-write in 2+ places → a shared helper candidate). + +Output a numbered list, each finding with the pattern, where it appears (file:line × N), and a proposed unifying shape. + +--- + +## Phase 3: Decisions log + +- **Architectural lock** (clear) → draft a `D#` for `~/src/graphrefly/decisions/decisions.jsonl`; append only after user approval, then update the DS-1 `locks` in `sessions/sessions.jsonl` and run `node ~/src/graphrefly/dashboard/build.mjs --check`. +- **Deferred / no clear answer** → append to `~/src/graphrefly/plan/backlog.jsonl` (B# + concrete trigger). +- **Recurring anti-pattern** → `~/src/graphrefly/plan/antipatterns.jsonl` (+ a `feedback_*` memory if generalizable). +- **Protocol-behavior change** surfaced → route to `/spec-amend` (spec-first), not a direct code change. + +--- + +## Output discipline + +- Be concrete. Quote `file:line` refs. +- Don't write "this looks good" — say WHICH Q5–Q9 dimensions clear and why. +- Don't pad. If Q6 has no caveats, write `**Hidden invariants:** none surfaced.` and move on. +- Don't second-guess the user's stated intent — Q8 alternatives are options to compare; Q5–Q7 probe the recommended shape. +- Skim-readable: headers per question, bullets within. + +--- + +## Authority hierarchy + +1. `~/src/graphrefly/spec/rules.jsonl` — the protocol 宪法. +2. `~/src/graphrefly/decisions/decisions.jsonl` (+ DS-1 narrative) — locked decisions + F-* floor + durable values. +3. `~/src/graphrefly/plan/phases.jsonl` — the CSP-* sequencer (phase locks). +4. `~/src/graphrefly/guide/guide.jsonl` (G-test / G-composition) — testability + composition shape. +5. Existing patterns in `packages/ts/src/` — only when the above are silent. + +If a finding conflicts with a higher-authority doc, surface it explicitly — DO NOT silently override (no-autonomous-decisions). + +--- + +## What to do AFTER this skill completes + +- Lock approved → `/dev-dispatch` (or `--light`) with the locked design; a protocol-behavior change goes through `/spec-amend` first. +- Decisions deferred → leave them in `backlog.jsonl` and move on. +- Needs more thought → HALT, summarize, let the user think. + +This skill produces a report; it modifies no implementation files (it only appends to `~/src/graphrefly` jsonl after explicit user approval of a `D#`). diff --git a/.agents/skills/dev-dispatch/SKILL.md b/.agents/skills/dev-dispatch/SKILL.md new file mode 100644 index 00000000..afc300cf --- /dev/null +++ b/.agents/skills/dev-dispatch/SKILL.md @@ -0,0 +1,123 @@ +--- +name: dev-dispatch +description: "Implement feature/fix with planning and self-test. Use when user says 'dispatch', 'dev-dispatch', or provides a task with implementation context. Supports --light flag for bug fixes and small changes. Run /qa afterward for code review and final checks." +argument-hint: "[--light] [task description or context]" +--- + +You are executing the **dev-dispatch** workflow for the **clean-slate GraphReFly** redesign. + +This repo is **`@graphrefly/ts`** — the self-contained TypeScript implementation (D32). Clean-slate code lands in **`packages/ts/src/`**. The language-neutral authority (spec / decisions / plan / conformance / formal) lives in **`~/src/graphrefly`** (branch `clean-slate`) as jsonl — when this skill and that repo disagree, **that repo wins** (AGENTS.md). Sibling impls: `@graphrefly/rust` (`~/src/graphrefly-rs`), `@graphrefly/py` (`~/src/graphrefly-py`) — each self-contained; cross-language = wire bridge, never in-process (D32). + +> **Stale-infra guard.** Do NOT reach for the retired port-model surfaces: `packages/pure-ts/**` (frozen read-only reference only, D41), `docs/implementation-plan.md`, `docs/optimizations.md`, `docs/roadmap.md`, `docs/test-guidance.md`, `docs/docs-guidance.md`, `GRAPHREFLY-SPEC.md`/`COMPOSITION-GUIDE.md` (migrated to `spec/rules.jsonl` + `guide/guide.jsonl`, B7), the `Impl`/facade/actor model / 3-digit D### port decisions. The clean-slate authority is the jsonl below. + +The user's task/context is: $ARGUMENTS + +### Mode detection +If `$ARGUMENTS` contains `--light`, this is **light mode**. Otherwise **full mode**. Differences are noted inline per phase. + +### Workflow floor (non-negotiable) +- **decision-first**: any architectural lock needs a `D#` in `~/src/graphrefly/decisions/decisions.jsonl` BEFORE code (`/design-review` → user approval → append). Decisions locked ≠ implementation approved — wait for an explicit "implement". +- **spec-first** (F-NO-IMPL-DEFINED): any wave-protocol behavior change amends `spec/rules.jsonl` + `formal/*.tla` + `spec/conformance.jsonl` FIRST (`/spec-amend`), THEN code. Operators/sugar/inspection are per-language (D6/D24) — NOT spec, skip spec-amend. +- **no autonomous decisions**: surface spec↔code conflicts; don't silently pick. File-by-file review for multi-file rewrites. +- **verify premise**: design tables lag code — grep the named symbols + check landed markers (`plan/phases.jsonl` status/notes) before designing new surface; a stale premise is a HALT. +- **consistency gate**: after touching any `~/src/graphrefly` jsonl, run `node ~/src/graphrefly/dashboard/build.mjs --check` (non-zero on broken links / orphans). + +--- + +## Phase 1: Context & Planning + +Load context and plan in a single pass. **Parallelize all reads.** + +Read in parallel (clean-slate authority): +- `~/src/graphrefly/AGENTS.md` — the single-source authority index (read FIRST). +- `~/src/graphrefly/spec/rules.jsonl` — the protocol 宪法 (R-* rules); deep-read the rules your change touches. +- `~/src/graphrefly/decisions/decisions.jsonl` — the unified D# log (or invoke `/decision-guard` to recall the governing D#/values/floor). +- `~/src/graphrefly/plan/phases.jsonl` — the CSP-* sequencer: find the phase this task belongs to, its `status` (done/impl/design), deps, and note. Read this FIRST among the plan files so you know whether you're on a ready phase or one still gated. +- `~/src/graphrefly/plan/backlog.jsonl` + `plan/antipatterns.jsonl` — deferred carries (B#) with triggers; anti-patterns to flag against. +- `~/src/graphrefly/spec/conformance.jsonl` — the behavioral scenarios (C-*) your change must keep green; check the `runtimes` status for the arm you target. +- `~/src/graphrefly/guide/guide.jsonl` — composition / test / docs / contribute guidance (G-composition / G-test / G-docs / G-contribute). +- `~/src/graphrefly/sessions/active/SESSION-clean-slate-redesign.md` (DS-1) — the L0–L6 design narrative + F-* constraints, when you need the why behind a lock. +- Any files the user referenced in $ARGUMENTS. +- The clean-slate source you'll modify: substrate = `packages/ts/src/{node,dispatcher,ctx,protocol,batch}/`; graph-layer = `packages/ts/src/graph/` (Graph + 8-verb sugar + operators + inspection describe/observe/profile). +- Existing tests: `packages/ts/src/__tests__/`. + +**Frozen reference (D41):** `packages/pure-ts/**` and `~/src/callbag-recharge` are READ-ONLY prior art for analogous operator behavior / edge cases / test structure during a re-derive (D40 Catalog-first). Map concepts to the clean-slate substrate (`node`/`ctx.down`/`ctx.depRecords`/`Graph`, D39 `describe`/`observe`) — do NOT 1:1 port; the old substrate API and semantics differ. They are NOT the behavior authority — `spec/rules.jsonl` is. + +While planning, validate proposed changes against the clean-slate floor (cite the rule/D#): +- **Sacred (L0.7):** topology declarative/serializable/inspectable · wave protocol is a public spec · wave protocol impl is **sync** · all fn go through the dispatcher. +- **8 verbs, closed (D4):** `node`/`graph`/`batch`/`state` + `producer`/`derived`/`effect`/`mount`. **Operators are `node` sugar (D6), not verbs** — per-language, never in parity (D24); real factory names show in `describe`. Adding a verb is a constitutional change. +- **Messages** `[[Type, Data?], ...]`; one array to `ctx.down`/`ctx.up` = one wave (R-msg-format). 10-type closed set, no user-defined types (R-msg-closed-set). +- **DIRTY before DATA/RESOLVED** in the same wave (R-dirty-before-data); two-phase glitch-free diamond (R-two-phase); a diamond/fan-in node recomputes exactly once after all changed deps settle (R-diamond). batch defers tier-≥3, not DIRTY. +- **`ctx.up` is control-tier only** (DIRTY/PAUSE/RESUME/INVALIDATE/TEARDOWN); DATA/RESOLVED/COMPLETE/ERROR are down-only (R-ctx-up, D8). A handle is pure data, no methods (D7). +- **No polling** (R-no-polling); **no imperative triggers** (R-no-imperative — reactive `ctx.up`/signals, not emitters/callbacks/timers+set; remove imperative paths when no caller depends); **no raw async** in the sync core (R-no-raw-async / F-SYNC-CORE — async lives only in sources / the pool / the wire bridge). +- **All fn through the dispatcher** (R-dispatch-all / F-DISPATCH-ALL — no inline-fn bypass). `dispatcher.invoke` is sync void (R-sync-core). +- **Data moves via messages** (R-data-not-peek — never peek a dep's `.cache` to seed compute; `.cache` is a read-only accessor for external consumers). SENTINEL = absence-of-DATA (R-sentinel); the canonical never-emitted detector is `ctx.prevData[i] === undefined`. +- **messageTier is a compile-time const table** (D18/D34/R-tier); the clock is **graph-local** (no global singleton, D26/R-clock); `onMessage`/`onSubscribe` are substrate-fixed, not user-replaceable (D19). +- **Primary-API clean** (R-primary-api-clean): protocol internals (DIRTY/RESOLVED/bitmask) never surface in value-level sugar (derived/effect/operator); ctx-level (node/producer) intentionally exposes tier as a power surface (DR-1). Sugar value-fn → ctx-fn wrapping happens in the graph layer (D27); a value-level `throw` becomes `[[ERROR,e]]` down (D30). +- **graph = single-thread causal/concurrency domain (D22 / R-graph-domain):** parallelism via pool callback or multi-graph + wire bridge; rewire is intra-graph only. +- **`ctx.state`** = per-node private cross-wave state (R-ctx-state); shared/observable state must be an explicit node + dep, not `ctx.state` (D23). A synchronous feedback cycle (a fn re-driving its own dep mid-wave) is a wave-level ERROR (D37/R-reentrancy), not iteration. +- **F-NO-WEDGE-CUT:** every primitive serves ≥2 segments (no LLM-only or single-segment wedge; F-NO-LLM-ONLY). **F-PERF:** budget every abstraction (thin node, default-off inspection). + +**Targeting a sibling (py/rust):** if the task targets `@graphrefly/py` (`~/src/graphrefly-py`) or `@graphrefly/rust` (`~/src/graphrefly-rs`), read that package's local layout + its conformance arm status in `spec/conformance.jsonl`. The cross-language contract is **behavioral conformance (D24)**, not symbol parity. PY public APIs are synchronous (return `Node[T]`/`Graph`/value, no `async def`); async lives at the source/pool boundary only (F-SYNC-CORE). + +Do NOT start implementing yet. + +--- + +## Phase 2: Architecture Discussion + +### Full mode — HALT + +**HALT and report before implementing.** Present: + +1. **Architecture assumptions** — how this fits the substrate (`node`/`dispatcher`/`ctx`/`protocol`/`batch`) vs graph-layer (`graph/`) split. +2. **New patterns** — any not yet in `packages/ts/src/`. +3. **Options considered** — alternatives with pros/cons. +4. **Recommendation** — preferred approach + why. + +Prioritize (in order): +1. **Correctness** — matches `~/src/graphrefly/spec/rules.jsonl` + the floor. +2. **Completeness** — edge cases (errors, COMPLETE, reconnect/reactivate, diamonds, SENTINEL gate, PAUSE lockset). +3. **Consistency** — matches patterns already in `packages/ts/src/`. +4. **Simplicity** — minimal solution. + +No backward compatibility (pre-1.0). + +**Escalation routing** (don't silently pick — no-autonomous-decisions): +- Architectural lock → `/design-review` → user approval → append a `D#` to `decisions.jsonl`. +- Wave-protocol behavior change → `/spec-amend` (spec-first: rules + TLA+ + conformance, THEN code). +- Cross-runtime concern → `/conformance` (behavioral scenario, not structural diff). +- Deferred/open question with no answer yet → append to `~/src/graphrefly/plan/backlog.jsonl` (B# + trigger); a recurring anti-pattern → `plan/antipatterns.jsonl` (+ a `feedback_*` memory if generalizable). + +**Wait for user approval before proceeding.** + +### Light mode — Skip unless escalation needed + +Proceed directly to Phase 3 **unless** Phase 1 reveals any of these: +- A change to **wave-protocol behavior** (tiers, wave semantics, diamond/equals/SENTINEL, batch, push-on-subscribe, ctx.up/down contract) → spec-first, escalate. +- A new architectural lock with no governing `D#`. +- Multiple viable approaches with non-obvious trade-offs. + +If any apply: HALT and present findings as in full mode. + +--- + +## Phase 3: Implementation & Self-Test + +After user approves (full mode) or after Phase 1 (light mode, no escalation): + +1. Implement the changes. + - Treat `~/src/graphrefly/spec/rules.jsonl` as non-negotiable for behavior; if code drifts from a rule, align to the rule — or surface the conflict, don't silently pick. + - Cite the governing R-id / D# in test expectations. +2. Create tests (per `guide/guide.jsonl` G-test — unit / property / conformance layering): + - Put tests in the most specific existing file under `packages/ts/src/__tests__/`. + - Use `graph.observe()` for live message assertions; assert at the node + message level otherwise. A behavioral-protocol change ALSO needs a `spec/conformance.jsonl` scenario (`/conformance`) before its rule flips `draft → active`. +3. Run checks: + - **TS:** `pnpm --filter @graphrefly/ts test` (vitest) + `pnpm run lint` (biome + layer/typecheck gates) + `pnpm run build` (tsup) as relevant. + - **PY (if targeted):** the `@graphrefly/py` package's own test/lint/type gates in `~/src/graphrefly-py`. + - **jsonl touched:** `node ~/src/graphrefly/dashboard/build.mjs --check` (consistency gate). +4. Fix any failures. + +If implementation leaves an **open architectural decision** (deferred behavior, parity caveat, "needs spec" item), append it to `~/src/graphrefly/plan/backlog.jsonl` (B# + trigger) — NOT a docs file. If it **lands or advances a CSP-* phase**, update that phase's `status`/`note` in `~/src/graphrefly/plan/phases.jsonl`, flip any conformance-backed `draft` rule to `active` once its scenario is green per arm, then run the consistency gate. + +When done, briefly list files changed and new exports added. Then suggest running `/qa` for adversarial review and final checks. diff --git a/.agents/skills/graph-animation/SKILL.md b/.agents/skills/graph-animation/SKILL.md new file mode 100644 index 00000000..c57191b2 --- /dev/null +++ b/.agents/skills/graph-animation/SKILL.md @@ -0,0 +1,418 @@ +--- +name: graph-animation +description: "Create GraphReFly concept explanation videos using HyperFrames. Use when asked to generate animated diagrams of reactive graph topologies — node activations, START/DIRTY/DATA/COMPLETE message flow, diamond resolution, batch mode, operators, etc. Supports two modes: default = concept-explainer (captioned, beat-structured, 30–90s); `--mode ambient-hero` = silent looping landing-page background (8-stage canonical storyboard, ~42s seamless loop). Invokes /hyperframes for composition authoring and /gsap for deterministic animation. Run /hyperframes-cli for preview/render commands." +argument-hint: "[--mode ambient-hero] [concept to animate, e.g. 'diamond resolution', 'batch mode', 'node lifecycle']" +--- + +You are executing the **graph-animation** workflow for **GraphReFly**. + +The user's request: `$ARGUMENTS` + +This skill produces a HyperFrames HTML composition that animates GraphReFly reactive graph concepts — topology diagrams with animated message flow, node activations, and tier-coded signals. It wraps `/hyperframes` and `/gsap` with GraphReFly-specific knowledge so you don't have to explain the domain each time. + +--- + +## Mode detection + +If `$ARGUMENTS` contains `--mode ambient-hero` (or the user asks for a "hero", "landing-page", "background video", or "loop"), use **ambient-hero mode** — skip directly to the "Mode B: ambient hero" section near the bottom. The 8-stage canonical storyboard is fully specified there; do not re-derive it. + +Otherwise, treat the request as **concept-explainer mode** (default) and follow Steps 1–4 below. + +--- + +## Step 1 — Understand the concept (concept-explainer mode only) + +If `$ARGUMENTS` is vague or empty, ask the user ONE question: which concept or workflow do you want to animate? Give them these options as examples: + +- Node lifecycle (START → DATA → COMPLETE → TEARDOWN) +- Message propagation through a linear chain (A → B → C) +- Diamond resolution (fan-out + fan-in, single recomputation) +- Batch mode (DIRTY cascades, DATA deferred until batch end) +- Reactive timer source feeding a derived node +- Operator pipeline (map → filter → combine) +- Human-in-the-loop gate (promptNode / valve pattern) +- Multi-agent subgraph ownership (L0–L3 staircase) + +If `$ARGUMENTS` names a concept already, proceed directly to Step 2. + +--- + +## Step 2 — GraphReFly visual language + +Use these conventions consistently across all GraphReFly animation videos. + +### Node shapes (canonical sugar names — `graph.state/producer/derived/effect`) + +| Sugar | Role | Shape | Color | +|---|---|---|---| +| `state` | Mutable cell, no deps | Circle | `#14B8A6` (teal) | +| `producer` | Push source (timer, event, async) | Diamond / pill | `#10B981` (emerald) | +| `derived` | Pure fn of deps | Circle | `#6C63FF` (violet) | +| `effect` | Sink / side-effect | Rounded rect | `#F59E0B` (amber) | +| External / human | Out-of-graph input | Hexagon | `#64748B` (slate) | + +Border stroke: `2px solid rgba(255,255,255,0.2)`. Inactive fill: 30% opacity of the node color. Active fill: 100% opacity. The underlying primitive `node()` exists below these four sugars; videos should use sugar names for vocabulary teaching. + +### Message symbols (canonical — `core/messages.ts`) + +The full protocol exports `START / DIRTY / DATA / RESOLVED / COMPLETE / TEARDOWN / INVALIDATE / PAUSE / RESUME / ERROR`. For videos, use the **foundational subset** unless a specific concept requires more: + +| Symbol | Color | Visual | When to show | +|---|---|---|---| +| `START` | `#94A3B8` (slate-light) | Thin pulse from src→dst, edge brightens | First message on a new connection (handshake) | +| `DIRTY` | `#FBBF24` (yellow) | Dashed pulse traveling along edge | Upstream may have changed; dep marks dirty | +| `DATA` | `#34D399` (green) | Solid dot traveling along edge; triggers node fn on arrival | Value flowing | +| `RESOLVED` | `#60A5FA` (blue) | Ring ripple on destination node | Diamond-resolution wave only (skip in beginner videos) | +| `COMPLETE` | `#94A3B8` (slate) | Edge fades, dst node dims | Producer finished emitting | +| `TEARDOWN` | `#A78BFA` (purple) | Edge erases backwards from dst→src | Subscription cancelled / dispose | +| `BATCH_*` | `#A78BFA` (purple) | Bracket sweep across topology | Batch-mode beats | + +Default foundational subset for hero / intro videos: **`START → DIRTY → DATA → COMPLETE`**. Add `RESOLVED` only when teaching diamond resolution; add `TEARDOWN` only when teaching lifecycle. + +### Edge conventions + +- Directed arrows from producer to consumer. +- Edge color matches the tier of the in-flight message. +- Resting state: `rgba(255,255,255,0.15)` gray. +- Animate as SVG `` or a CSS `clip-path` wipe so timing is seekable. + +### Layout + +- Canvas: **1920×1080** (landscape), dark background `#0F172A`. +- Nodes: minimum 80px diameter circles, 24px sans-serif label inside. +- Edges: 3px stroke with arrowhead marker. +- Label overlay: bottom-left, `font-size: 28px`, `color: #E2E8F0`, describes what is happening. + +### Timing budget + +- 2–4 seconds per "beat" (one message hop or one lifecycle phase). +- Total target: **30–90 seconds** for a concept clip. +- Add 1.5s of static "intro frame" showing the graph topology before any animation begins. + +--- + +## Step 3 — Scaffold the composition + +Use `npx hyperframes init ` to scaffold a new project directory, OR create the file inline if the user prefers a single-file drop. + +A minimal starting template (adapt to the specific concept): + +```html + + + + + + + + + + + + +
+ + + + + + + + + + + + + + + + + +
+
+ + +
+
+ + + + +``` + +--- + +## Step 4 — Mode A: concept-explainer patterns + +### Diamond resolution + +``` + A + / \ + B C + \ / + D (recomputes once) +``` + +Sequence: +1. A emits DATA → DIRTY cascades to B and C simultaneously (two yellow dashes) +2. B and C each emit DIRTY to D +3. D receives both DIRTYs — stays dirty, does NOT recompute yet +4. A emits RESOLVED → B resolves → C resolves → D resolves +5. D recomputes **once** (green DATA dot appears from D) +6. Caption: "Diamond resolved — one recomputation, zero double-fires" + +### Batch mode + +Sequence: +1. `batch(() => ...)` bracket appears — purple "BATCH_START" ring +2. Multiple sources emit DATA — nodes flash DIRTY but DATA dots are held (shown as queued dots stacked at source) +3. batch end — all deferred DATA releases simultaneously +4. Caption: "Batch defers DATA, not DIRTY — downstream sees one coherent update" + +### Node lifecycle + +Sequence: +1. Node dims (initial) +2. Subscription arrives → node brightens +3. Upstream emits DATA → node pulses green +4. RESOLVED ripple out +5. Unsubscribe → teardown ring (purple) → node dims again + +### Operator pipeline + +Show 3–4 nodes in a horizontal chain (map → filter → combine). Animate a value token flowing left to right, label each node with its transform (`×2`, `> 5`, `merge`). + +--- + +## Step 5 — Run the dev loop + +```bash +# from the composition project directory: +npx hyperframes lint # validate timing/structure +npx hyperframes preview # live browser preview with hot reload +npx hyperframes render # output MP4 +``` + +Use `/hyperframes-cli` skill for detailed CLI guidance. +Use `/gsap` skill for advanced GSAP timeline patterns. +Use `/hyperframes-media` skill if you need TTS narration or audio. + +--- + +## Step 6 — Quality checks before render + +- [ ] All nodes have correct shape + color per the visual language table above +- [ ] Message colors match the canonical table (START=slate-light, DIRTY=yellow, DATA=green, COMPLETE=slate, RESOLVED=blue if shown, TEARDOWN=purple if shown) +- [ ] Timeline registers `window.__hfGsapTimeline = tl` (required for seekable render) +- [ ] Caption text is ≤ 80 chars per line and readable at 1080p (concept-explainer mode only) +- [ ] `npx hyperframes lint` passes with no errors +- [ ] Preview scrubbing doesn't stutter (all animations are on the GSAP timeline, not setTimeout) + +--- + +## Mode B: ambient hero (canonical 8-stage landing-page loop) + +This mode produces the **GraphReFly landing-page background video** — a silent, looping, atmospheric reactive-graph composition. Use it whenever the user asks for a "hero", "landing-page", "background video", or passes `--mode ambient-hero`. + +### Mode-B rules (differ from concept-explainer) + +| Rule | Value | Why | +|---|---|---| +| Length | ~42s seamless loop | Hero convention 15–60s; matches the 8-stage plan | +| Audio | none | Plays muted; hero conventions | +| Captions | none over the topology | Headline text overlays the hero on the page; captions would compete | +| Small floating labels | OK, ≤ 4 words, ≤ 22px font | E.g. `graph.describe()`, `graph.observe()`, `batch()` — vocabulary pre-teach | +| Pacing | slow / ambient | No frantic motion; eye should rest | +| Loop seam | **Stage 8 ends at exactly the same camera/node positions as Stage 1 starts** | Imperceptible loop | +| Contrast | reduced — palette at ~70% saturation, bg `#0F172A` | Headline text on page must dominate | +| Render output | MP4 + a WebM/VP9 variant for `