Skip to content

feat: add bounded nightly brain digest - #1397

Merged
mrubens merged 3 commits into
developfrom
codex/brain-daily-digest
Aug 16, 2026
Merged

feat: add bounded nightly brain digest#1397
mrubens merged 3 commits into
developfrom
codex/brain-daily-digest

Conversation

@mrubens

@mrubens mrubens commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

What changed

  • replace per-page dream reflection and pattern fan-out with one bounded nightly digest
  • search only the effective-date window since the last successful run
  • require citations to eligible primary pages and reject out-of-window sources
  • store one digest at daily/digests/YYYY-MM-DD
  • keep gbrain autopilot maintenance for embedding and index upkeep
  • document the new nightly behavior

Why

The generic reflection pipeline made hundreds of model calls while producing broad, low-value output. A single operational digest is cheaper, easier to inspect, and better aligned with the useful nightly synthesis we want.

Validation

  • focused BullMQ test suite (10 tests)
  • BullMQ typecheck
  • changed-file ESLint
  • deployment artifact validation
  • pre-push oxlint, fast typecheck, residual lint, and knip checks

@roomote-community

roomote-community Bot commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

1 issue outstanding. See task

  • apps/bullmq/src/scheduled-jobs/brain-maintenance.ts:238 The digest's effective-date source filter is not enforced.
  • apps/bullmq/src/scheduled-jobs/brain-maintenance.ts:278 Concurrent ingestion can be permanently skipped by the digest watermark.
  • apps/bullmq/src/scheduled-jobs/brain-maintenance.ts:266 Timestamp boundaries skip date-only source pages.

Reviewed 004db7b

: new Date(now.getTime() - BRAIN_DAILY_DIGEST_INITIAL_LOOKBACK_MS);
const searchBody = await callGbrainTool(readConnection, 'search', {
query: DAILY_DIGEST_SEARCH_QUERY,
since: since.toISOString(),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

search does not accept since or until in the pinned gbrain MCP contract, so these arguments are silently ignored. The downstream synthesize verb also uses its window only in its prompt, not its gather query. As a result, this candidate set can contain historical pages, and eligibleSlugs will accept citations outside the intended effective-date window. Use retrieval that actually filters by effective date, then validate the eligible pages before accepting citations.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching to query makes the retrieval filter real, but the new 06:00 timestamp bounds still drop date-only source pages. Roomote writes task pages as date: YYYY-MM-DD, which gbrain stores at midnight; its since filter is strict. On the Aug 17 pass, since=2026-08-16T06:00:00Z excludes every page effective-dated 2026-08-16, including work completed later that day, and the next watermark never revisits it. Use bounds compatible with date-granular effective dates (or persist and query a timestamp-granular source date), with a regression test for this case.

Comment thread apps/bullmq/src/scheduled-jobs/brain-maintenance.ts
@mrubens
mrubens marked this pull request as ready for review August 16, 2026 14:50
@mrubens
mrubens merged commit 2088d72 into develop Aug 16, 2026
18 checks passed
@mrubens
mrubens deleted the codex/brain-daily-digest branch August 16, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant