Skip to content

feat(ledger): scheduled checkpoint anchoring, tying Rekor and git-commit together - #9413

Merged
JSONbored merged 1 commit into
mainfrom
feat/ledger-anchor-scheduler
Jul 27, 2026
Merged

feat(ledger): scheduled checkpoint anchoring, tying Rekor and git-commit together#9413
JSONbored merged 1 commit into
mainfrom
feat/ledger-anchor-scheduler

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Sixth and final scheduled-side sub-issue of #9267 (external ledger anchoring), stacked on #9272 (#9397#9273's own PR #9403 has already merged into this base). Ties #9272 (Rekor) and #9273 (git-commit) together into a periodic job.

Enqueued every ~2min cron tick unconditionally (added anchor-decision-ledger to IMMEDIATE_SCHEDULED_JOB_TYPES, alongside the existing regate sweep) rather than gated to isHourly: the seq-threshold trigger must fire independent of the hourly clock per the issue's own requirement, and the job's own decideLedgerAnchorSchedule decides whether a given tick actually does anything — a cheap two-query no-op the overwhelming majority of ticks.

decideLedgerAnchorSchedule is pure: anchors on the hourly tick when the tip changed since the last attempt (never-anchored counts as "changed"), anchors immediately once the seq delta reaches 256 regardless of the clock (treating a never-anchored ledger as starting from seq 0, so a big first burst doesn't wait for the next hourly tick), skips an unchanged tip either way.

Both backends run independently per checkpoint. The "never rejects" safety boundary is enforced structurally, not by convention: submitToRekor/submitToGitAnchor never reject on their own, but the real git path wraps installation-token minting around submitToGitAnchor — a genuine IO call that can throw, unlike anything inside submitToGitAnchor itself — so the orchestrator catches each backend call centrally and records a status: 'failed' row on an unexpected rejection, rather than trusting every future caller to uphold the invariant.

Also found and removed real dead code: an old, unversioned {seq, rowHash, at} stub buildLedgerAnchorPayload was still sitting in decision-record.ts, alongside the new signed/versioned one #9270 built specifically to supersede it in ledger-anchor.ts. Same function name, two implementations — never cleaned up when #9270 landed. Removed the dead stub and its now-orphaned tests.

Closes #9274.

Test plan

  • npx vitest run test/unit/ledger-anchor-scheduler.test.ts — 21/21, 100% statement/branch/function/line coverage
  • Every scheduling permutation proven: hourly+changed, hourly+unchanged (skip), threshold-independent-of-clock, never-anchored-starts-from-0, threshold-takes-priority-over-hourly
  • Both "unconfigured" degrade paths (no current key, no private key) proven distinctly
  • The real default submitRekor path exercised (global-fetch stub), not just the injected fake
  • A rejecting submitGit (simulating a token-mint failure) proven to land as a recorded status: 'failed' row, not an unhandled rejection
  • npx vitest run test/unit/ledger-anchor-job-dispatch.test.ts — 2/2: the real dispatch wiring, including a real installation-token mint and the real submitToGitAnchor GET→PUT call sequence against a stubbed Contents API
  • npx vitest run test/unit/decision-record.test.ts test/unit/ledger-anchor-persistence.test.ts — new loadDecisionLedgerTip/loadLastLedgerAnchorAttempt covered directly in their own files
  • npx vitest run test/unit/index.test.ts test/unit/selfhost-queue-common.test.ts — all 10+ affected exact-array assertions updated and passing, confirming the new job type's presence/position/cadence across every tick shape (:00, :14, :30, 06:00, broker-only)
  • Full relevant suite together: 162+ passing, no regressions
  • npx tsc --noEmit clean · npm run db:schema-drift:check / db:migrations:check clean · npx tsx scripts/check-import-specifiers.ts clean

Base branch is feat/ledger-anchor-rekor (#9272's PR #9397) — #9273's own PR (#9403) already merged into this same base, so it's included here too. Merge order remaining: #9392#9397 (now includes #9273) → this.

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 27, 2026
@JSONbored
JSONbored force-pushed the feat/ledger-anchor-rekor branch from 7f4eda6 to 1c7a6b8 Compare July 27, 2026 16:59
Base automatically changed from feat/ledger-anchor-rekor to main July 27, 2026 17:05
@loopover-orb

loopover-orb Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-27 17:26:43 UTC

14 files · 1 AI reviewer · 1 blocker · CI green · clean

⏸️ Suggested Action - Manual Review

Review summary
This PR wires the previously-built Rekor (#9272) and git-commit (#9273) anchoring backends into a periodic scheduler: a pure decision function (decideLedgerAnchorSchedule) picks hourly-on-change or seq>=256-threshold anchoring, and the orchestrator centrally catches each backend call so an injected submitGit (which wraps a real installation-token mint that can throw) can never produce an unhandled rejection. It also correctly removes the genuinely dead, unversioned buildLedgerAnchorPayload stub left behind when #9270 replaced it with the signed/versioned payload builder in ledger-anchor.ts. The job-dispatch/env/index/types wiring is consistent end-to-end (config resolution, installation-token retry, IMMEDIATE_SCHEDULED_JOB_TYPES entry) and is backed by both a pure-logic test file and an end-to-end dispatch test that exercises a real token mint and Contents API call sequence.

Nits — 6 non-blocking
  • The external brief flags `test/unit/ledger-anchor-job-dispatch.test.ts:19` and `test/unit/ledger-anchor-scheduler.test.ts:111` as potential leaked private_key material — these are freshly `crypto.subtle.generateKey`-generated ephemeral test keypairs (via `generatePrivateKeyPem`/`generateAnchorKeypair`), not real secrets, but worth a quick visual confirmation.
  • `src/review/ledger-anchor-scheduler.ts:87` logs via `console.log` on the unconfigured-skip path — consistent with the rest of the codebase's structured-log convention, so not a real issue, just flagged for visibility.
  • The magic number `256` (LEDGER_ANCHOR_SEQ_THRESHOLD) is already exported as a named constant at ledger-anchor-scheduler.ts:16, so the brief's 'magic number' flag on that file is a false positive; only the persistence file's use of literal `9274` in a comment is genuinely just an issue reference, not a magic number.
  • `decideLedgerAnchorSchedule` (ledger-anchor-scheduler.ts) has a few nested conditionals but stays readable; not worth restructuring for a one-time depth-5 flag.
  • Per repo convention, confirm this PR closes/links an eligible open issue — the description ties it to ledger: scheduled checkpoint anchoring — hourly or every 256 records, whichever first #9274 (epic orb(trust): external decision-ledger anchoring — Rekor + git-commit, from tamper-evident to tamper-proof #9267), which satisfies that requirement here.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

Concerns raised — review before merging

  • Possible leaked secret in the diff (private_key_block): The PR diff matches secret pattern(s): private_key_block. Found at: test/unit/ledger-anchor-job-dispatch.test.ts:19, test/unit/ledger-anchor-scheduler.test.ts:111. A committed credential must be rotated and removed from the change before merge. — Remove the secret from the diff, rotate the exposed credential, then re-run the gate.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. Possible leaked secret in the diff (private_key_block): The PR diff matches secret pattern(s): private_key_block. Found at: test/unit/ledger-anchor-job-dispatch.test.ts:19, test/unit/ledger-anchor-scheduler.test.ts:111. A committed credential must be rotated and removed from the change before merge. — Remove the secret from the diff, rotate the exposed credential, then re-run the gate.

Decision drivers

  • ❌ Code review — 1 blocker (1 reviewer)
  • ❌ Gate result — Blocking (Repo-configured hard blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #9274
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 13 registered-repo PR(s), 13 merged, 345 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 13 PR(s), 345 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Linked issue satisfaction

Addressed
The PR wires a scheduled anchoring job into the existing cron dispatch (no new trigger), implements the pure hourly/skip-if-unchanged/≥256-seq-threshold decision logic, runs both Rekor and git backends independently via Promise.all with per-backend failure isolation recorded through the #9271 persistence layer, and the job has no caller in the decision-record persist path so it cannot block a revi

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, Ruby, Go, MDX, Shell, Solidity, JavaScript
  • Official Gittensor activity: 13 PR(s), 345 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: hold · clause: secret_leak
  • config: 6c1f5fb2375b15a13e90037e4cd8fe13dd6999f4e13d58e7f588ba39b5c6a7e8 · pack: oss-anti-slop · ci: passed
  • record: 79c19d9ce49ebfbbd970672cb6fb76f7dd1905eb3215ba93ad38868e59d2f9d1 (schema v5, head 505ffba)

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@JSONbored
JSONbored force-pushed the feat/ledger-anchor-scheduler branch from 7079aca to c0a3874 Compare July 27, 2026 17:06
…mit together (#9274)

Sixth and final scheduled-side sub-issue of #9267. Enqueued every ~2min cron
tick unconditionally (added to IMMEDIATE_SCHEDULED_JOB_TYPES alongside the
regate sweep) rather than gated to isHourly: the seq-threshold trigger must
fire independent of the hourly clock per the issue's own requirement, and the
job's own decideLedgerAnchorSchedule is what actually decides whether a given
tick does anything — a cheap two-query no-op the overwhelming majority of
ticks.

decideLedgerAnchorSchedule is pure: anchors on the hourly tick when the tip
changed since the last attempt (never-anchored counts as "changed"), anchors
immediately once the seq delta reaches 256 regardless of the clock (treating a
never-anchored ledger as starting from seq 0, so a big first burst doesn't
wait for the next hourly tick), and skips an unchanged tip either way.

Both backends run independently per checkpoint. The safety boundary is
structural, not conventional: submitToRekor and submitToGitAnchor never reject
on their own, but the REAL git path wraps installation-token minting around
submitToGitAnchor (a genuine IO call that can throw, unlike anything inside
submitToGitAnchor itself) — so the orchestrator catches each backend call
centrally and records a status:'failed' row on an unexpected rejection, rather
than trusting every future caller to uphold "never rejects" by convention.

Also found and removed real dead code: the OLD unversioned {seq, rowHash, at}
stub buildLedgerAnchorPayload, still sitting in decision-record.ts alongside
the new signed/versioned one #9270 built to supersede it in ledger-anchor.ts.
Never removed when #9270 landed — same function name, two implementations,
the old one dead and untested by anything except its own now-removed tests.

Added loadDecisionLedgerTip (decision-record.ts) and loadLastLedgerAnchorAttempt
(ledger-anchor-persistence.ts) as the two reads the scheduler needs — the
latter deliberately keyed on the last ATTEMPT regardless of status, not the
last success, so a backend stuck failing doesn't get hammered against the same
stale checkpoint forever while newer tips go unanchored.

Updates ~10 existing exact-array assertions in index.test.ts and one in
selfhost-queue-common.test.ts to include the new job type — real, necessary
churn from enqueuing unconditionally on every tick, not routed around.
@JSONbored
JSONbored force-pushed the feat/ledger-anchor-scheduler branch from c0a3874 to 505ffba Compare July 27, 2026 17:08
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.63%. Comparing base (6a7c1af) to head (505ffba).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9413      +/-   ##
==========================================
- Coverage   89.52%   88.63%   -0.90%     
==========================================
  Files         842      843       +1     
  Lines      109835   109880      +45     
  Branches    26159    26173      +14     
==========================================
- Hits        98333    97387     -946     
- Misses      10239    11521    +1282     
+ Partials     1263      972     -291     
Flag Coverage Δ
backend 93.60% <100.00%> (-1.65%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/index.ts 96.46% <100.00%> (+0.03%) ⬆️
src/queue/job-dispatch.ts 99.48% <100.00%> (+0.02%) ⬆️
src/review/decision-record.ts 100.00% <100.00%> (ø)
src/review/ledger-anchor-persistence.ts 100.00% <100.00%> (ø)
src/review/ledger-anchor-scheduler.ts 100.00% <100.00%> (ø)
src/selfhost/queue-common.ts 98.65% <ø> (ø)
src/types.ts 100.00% <ø> (ø)

... and 3 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 27, 2026
@JSONbored
JSONbored merged commit 6faa9a8 into main Jul 27, 2026
7 checks passed
@JSONbored
JSONbored deleted the feat/ledger-anchor-scheduler branch July 27, 2026 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ledger: scheduled checkpoint anchoring — hourly or every 256 records, whichever first

1 participant