Skip to content

feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes#245

Open
iceteaSA wants to merge 2 commits into
cortexkit:masterfrom
iceteaSA:cache-ttl-never
Open

feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes#245
iceteaSA wants to merge 2 commits into
cortexkit:masterfrom
iceteaSA:cache-ttl-never

Conversation

@iceteaSA

@iceteaSA iceteaSA commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Problem

cache_ttl assumes idle > TTL means the provider evicted the prompt cache, so the next prefix rebuild is free. Two consumers act on that assumption:

  1. the scheduler converts a defer pass into an execute pass (scheduler.ts), and
  2. mustMaterialize fires the ttl_idle HARD fold (via hardCacheExpired in transform.ts).

On lanes kept warm by an external keepwarm mechanism (prewarm proxies, dedicated cache-keep tools that re-warm the provider cache out-of-band), the assumption is wrong: the cache never goes cold, MC's lastResponseTime goes stale anyway, and both consumers false-positive. MC then initiates a rebuild it believes is free that is actually a full paid cache-write — measured 450-560K tokens per fold on large sessions. There was no way to express "this lane never goes cold": the config requires a duration.

Fix

cache_ttl: "never" (case-insensitive, works as the string form or any per-model value):

  • parseCacheTtl("never") returns Infinity; both consumers go inert through the existing comparisons (elapsed > Infinity / elapsed >= Infinity are never true) — no new branches in the hot path.
  • The Rust scheduler mirrors the sentinel with u64::MAX (its predicates already use saturating_sub, so no overflow path).
  • Status surfaces render honestly instead of a bogus countdown: /ctx-status shows never expires (always-warm lane); the TUI sidebar gets a JSON-safe cacheNeverExpires flag on StatusDetail (Infinity does not survive JSON-RPC); Pi's status dialog now uses the shared parseCacheTtl (it previously used a private fallback parser that would have shown a 5m countdown).
  • hardCacheExpired is extracted into a pure computeHardCacheExpired helper so the "never" -> Infinity -> false chain has direct unit coverage instead of only flag-consumption coverage.

Docs: CONFIGURATION.md describes the sentinel, what it disables, and the tradeoff — after a genuinely cold start (e.g. the keepwarm process died), the free-fold window is not detected on such lanes; mutations then apply at the execute threshold.

Verification

  • plugin: 2995 pass / 0 fail; pi-plugin: 666 pass / 0 fail; tsc --noEmit clean on both
  • new tests red-verified: reverting the sentinel fails the scheduler tests and the computeHardCacheExpired test
  • Rust: sentinel + predicate + scheduler-decision tests added in scheduler.rs's test module (note: I could not run cargo test locally — the crates workspace has path deps outside this repo; the change is a 3-line early return mirroring the TS logic)
  • schema + generated docs regenerated (build-schema / build-config-docs drift tests green); check:tui-compiled green

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Adds cache_ttl: "never" to disable the idle-TTL heuristic on always-warm lanes, preventing false executes and costly cache rebuilds. Also fixes status JSON for always‑warm lanes and restores invalid‑TTL diagnostics.

  • New Features

    • parseCacheTtl("never") returns Infinity (TS) / u64::MAX (Rust), so idle-execute and ttl_idle never fire; computeHardCacheExpired added for consistent checks with tests.
    • Status UIs show “never expires” with cacheNeverExpires; Pi uses shared parser; docs and schema updated.
  • Bug Fixes

    • Status RPC: keep cacheRemainingMs JSON-safe (no Infinity); use 0 plus cacheNeverExpires.
    • Transform: computeHardCacheExpired now accepts onInvalid; restores the invalid-TTL pass outcome and session log.

Written for commit 620c138. Summary will update on new commits.

Review in cubic

Greptile Summary

Adds a JSON-safe "never" cache-TTL sentinel for always-warm lanes.

  • Maps "never" to an infinite TTL in the TypeScript and Rust schedulers, disabling idle-triggered execution and hard folds.
  • Preserves invalid-TTL diagnostics while sharing TTL parsing across transform and status paths.
  • Adds an explicit RPC flag and updates status interfaces to render never-expiring caches correctly.
  • Updates schemas, documentation, generated TUI code, and tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failures remain.

Important Files Changed

Filename Overview
packages/plugin/src/features/magic-context/scheduler.ts Adds case-insensitive parsing of the "never" sentinel as positive infinity.
crates/mc-module/src/scheduler.rs Mirrors the sentinel with u64::MAX and adds scheduler and predicate coverage.
packages/plugin/src/hooks/magic-context/transform.ts Extracts the hard-cache-expiry calculation while retaining invalid-TTL diagnostics and fallback behavior.
packages/plugin/src/plugin/rpc-handlers.ts Encodes never-expiring cache status with a JSON-safe flag and finite numeric fields.
packages/plugin/src/shared/rpc-types.ts Extends status details with the optional cacheNeverExpires compatibility flag.
packages/plugin/src/tui/index.tsx Uses the explicit status flag to render never-expiring TTL and threshold-only execution behavior.
packages/pi-plugin/src/dialogs/status-dialog.ts Reuses shared TTL parsing and displays the always-warm sentinel without a countdown.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[cache_ttl: never] --> B[Shared TTL parser]
    B --> C[TypeScript: Infinity]
    B --> D[Rust: u64::MAX]
    C --> E[Idle scheduler remains deferred]
    C --> F[Hard TTL fold remains inactive]
    C --> G[RPC status conversion]
    G --> H[cacheNeverExpires: true]
    G --> I[JSON-safe numeric fields]
    H --> J[TUI renders never expires]
    D --> K[Rust idle predicates remain false]
Loading

Reviews (2): Last reviewed commit: "fix(scheduler): keep cacheRemainingMs JS..." | Re-trigger Greptile

Lanes kept warm by external keepwarm mechanisms (prewarm proxies,
dedicated cache-keep tools) re-warm the provider prompt cache
out-of-band, so MC's idle>TTL heuristic false-positives: both TTL
consumers (scheduler idle-execute and the ttl_idle m[0] fold) initiate
a rebuild believed free that is actually a full paid cache-write
(measured 450-560K tokens on large sessions).

cache_ttl: "never" (string or per-model value, case-insensitive)
disables both consumers: parseCacheTtl returns Infinity, so
elapsed>ttl / elapsed>=ttl never fire. Rust scheduler mirrors the
sentinel with u64::MAX (predicates already use saturating_sub).
Status surfaces render "never expires (always-warm lane)" instead of
a bogus countdown: /ctx-status, the TUI sidebar (JSON-safe
cacheNeverExpires flag on StatusDetail), and Pi's status dialog
(which previously parsed the TTL with a private fallback parser).
hardCacheExpired extracted to a pure computeHardCacheExpired helper
with direct coverage of the never-chain.

Tradeoff documented in CONFIGURATION.md: on a genuinely cold start
the free-fold window is not detected on such lanes; mutations then
apply at the execute threshold.
Comment thread packages/plugin/src/plugin/rpc-handlers.ts Outdated
Comment thread packages/plugin/src/hooks/magic-context/transform.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/plugin/src/hooks/magic-context/transform.ts
Comment thread packages/plugin/src/plugin/rpc-handlers.ts Outdated
…TL diagnostics

- rpc-handlers: the cacheNeverExpires branch assigned Infinity to
  cacheRemainingMs; JSON.stringify converts Infinity to null over RPC,
  violating the numeric StatusDetail contract. Use 0 and let the
  cacheNeverExpires flag carry the semantics (the TUI keys on it first).
- computeHardCacheExpired: the extraction dropped the
  invalid-cache-ttl-fallback pass outcome and session log on parse
  failure. Add an onInvalid callback; the transform call site restores
  the exact pre-extraction record + log.
- test: seed last_response_time in the never-TTL status test — the
  guarded branch only runs when lastResponseTime > 0, so the assertion
  was vacuous without it (verified red: Infinity revert now fails it).
@iceteaSA

Copy link
Copy Markdown
Contributor Author

Note on the red Check (dashboard): it's pre-existing on master, not introduced here — this PR touches no dashboard files. The same 2 ConfigEditor ⇄ schema parity failures reproduce on a pristine upstream/master checkout at v0.33.0 (48ab531d):

- []
+ [
+   "experimental.mural.enabled",
+   "experimental.mural.model",
+   "fail_closed_blocking",
+   "pi.subagent_extensions",
+ ]

The v0.33.0 schema added those four leaves without updating the dashboard's RENDERED_PREFIXES/OMITTED_BY_DESIGN lists, and experimental.mural.* also trips the "nothing re-introduces the dead experimental.* namespace" test. Happy to send a separate fix PR for the parity lists if useful — the right classification (render vs omit-by-design, and whether mural should live under experimental.) is a maintainer call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant