feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes#245
feat(scheduler): add cache_ttl "never" sentinel for always-warm lanes#245iceteaSA wants to merge 2 commits into
Conversation
Lanes kept warm by external keepwarm mechanisms (prewarm proxies, dedicated cache-keep tools) re-warm the provider prompt cache out-of-band, so MC's idle>TTL heuristic false-positives: both TTL consumers (scheduler idle-execute and the ttl_idle m[0] fold) initiate a rebuild believed free that is actually a full paid cache-write (measured 450-560K tokens on large sessions). cache_ttl: "never" (string or per-model value, case-insensitive) disables both consumers: parseCacheTtl returns Infinity, so elapsed>ttl / elapsed>=ttl never fire. Rust scheduler mirrors the sentinel with u64::MAX (predicates already use saturating_sub). Status surfaces render "never expires (always-warm lane)" instead of a bogus countdown: /ctx-status, the TUI sidebar (JSON-safe cacheNeverExpires flag on StatusDetail), and Pi's status dialog (which previously parsed the TTL with a private fallback parser). hardCacheExpired extracted to a pure computeHardCacheExpired helper with direct coverage of the never-chain. Tradeoff documented in CONFIGURATION.md: on a genuinely cold start the free-fold window is not detected on such lanes; mutations then apply at the execute threshold.
There was a problem hiding this comment.
All reported issues were addressed across 20 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…TL diagnostics - rpc-handlers: the cacheNeverExpires branch assigned Infinity to cacheRemainingMs; JSON.stringify converts Infinity to null over RPC, violating the numeric StatusDetail contract. Use 0 and let the cacheNeverExpires flag carry the semantics (the TUI keys on it first). - computeHardCacheExpired: the extraction dropped the invalid-cache-ttl-fallback pass outcome and session log on parse failure. Add an onInvalid callback; the transform call site restores the exact pre-extraction record + log. - test: seed last_response_time in the never-TTL status test — the guarded branch only runs when lastResponseTime > 0, so the assertion was vacuous without it (verified red: Infinity revert now fails it).
|
Note on the red The v0.33.0 schema added those four leaves without updating the dashboard's |
Problem
cache_ttlassumes idle > TTL means the provider evicted the prompt cache, so the next prefix rebuild is free. Two consumers act on that assumption:scheduler.ts), andmustMaterializefires thettl_idleHARD fold (viahardCacheExpiredintransform.ts).On lanes kept warm by an external keepwarm mechanism (prewarm proxies, dedicated cache-keep tools that re-warm the provider cache out-of-band), the assumption is wrong: the cache never goes cold, MC's
lastResponseTimegoes stale anyway, and both consumers false-positive. MC then initiates a rebuild it believes is free that is actually a full paid cache-write — measured 450-560K tokens per fold on large sessions. There was no way to express "this lane never goes cold": the config requires a duration.Fix
cache_ttl: "never"(case-insensitive, works as the string form or any per-model value):parseCacheTtl("never")returnsInfinity; both consumers go inert through the existing comparisons (elapsed > Infinity/elapsed >= Infinityare never true) — no new branches in the hot path.u64::MAX(its predicates already usesaturating_sub, so no overflow path)./ctx-statusshowsnever expires (always-warm lane); the TUI sidebar gets a JSON-safecacheNeverExpiresflag onStatusDetail(Infinity does not survive JSON-RPC); Pi's status dialog now uses the sharedparseCacheTtl(it previously used a private fallback parser that would have shown a 5m countdown).hardCacheExpiredis extracted into a purecomputeHardCacheExpiredhelper so the"never" -> Infinity -> falsechain has direct unit coverage instead of only flag-consumption coverage.Docs:
CONFIGURATION.mddescribes the sentinel, what it disables, and the tradeoff — after a genuinely cold start (e.g. the keepwarm process died), the free-fold window is not detected on such lanes; mutations then apply at the execute threshold.Verification
tsc --noEmitclean on bothcomputeHardCacheExpiredtestscheduler.rs's test module (note: I could not runcargo testlocally — the crates workspace has path deps outside this repo; the change is a 3-line early return mirroring the TS logic)build-schema/build-config-docsdrift tests green);check:tui-compiledgreenNeed help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.Summary by cubic
Adds
cache_ttl: "never"to disable the idle-TTL heuristic on always-warm lanes, preventing false executes and costly cache rebuilds. Also fixes status JSON for always‑warm lanes and restores invalid‑TTL diagnostics.New Features
parseCacheTtl("never")returns Infinity (TS) /u64::MAX(Rust), so idle-execute andttl_idlenever fire;computeHardCacheExpiredadded for consistent checks with tests.cacheNeverExpires; Pi uses shared parser; docs and schema updated.Bug Fixes
cacheRemainingMsJSON-safe (no Infinity); use 0 pluscacheNeverExpires.computeHardCacheExpirednow acceptsonInvalid; restores the invalid-TTL pass outcome and session log.Written for commit 620c138. Summary will update on new commits.
Greptile Summary
Adds a JSON-safe
"never"cache-TTL sentinel for always-warm lanes."never"to an infinite TTL in the TypeScript and Rust schedulers, disabling idle-triggered execution and hard folds.Confidence Score: 5/5
The PR appears safe to merge.
No blocking failures remain.
Important Files Changed
"never"sentinel as positive infinity.u64::MAXand adds scheduler and predicate coverage.cacheNeverExpirescompatibility flag.Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR A[cache_ttl: never] --> B[Shared TTL parser] B --> C[TypeScript: Infinity] B --> D[Rust: u64::MAX] C --> E[Idle scheduler remains deferred] C --> F[Hard TTL fold remains inactive] C --> G[RPC status conversion] G --> H[cacheNeverExpires: true] G --> I[JSON-safe numeric fields] H --> J[TUI renders never expires] D --> K[Rust idle predicates remain false]Reviews (2): Last reviewed commit: "fix(scheduler): keep cacheRemainingMs JS..." | Re-trigger Greptile