Skip to content

feat(ledger): git-commit anchoring backend, cross-verified via GH Archive - #9403

Merged
loopover-orb[bot] merged 1 commit into
feat/ledger-anchor-rekorfrom
feat/ledger-anchor-git
Jul 27, 2026
Merged

feat(ledger): git-commit anchoring backend, cross-verified via GH Archive#9403
loopover-orb[bot] merged 1 commit into
feat/ledger-anchor-rekorfrom
feat/ledger-anchor-git

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Fifth sub-issue of #9267 (external ledger anchoring), stacked on #9272 (#9397). Secondary, complementary to Rekor: appends one JSONL line per anchor to a public repo via the GitHub Contents API and the existing makeInstallationOctokit chokepoint — never a direct fetch to the GitHub API, matching every other GitHub write in this engine.

Alone this is weaker than Rekor (GitHub is a trusted third party, git push --force rewrites it). It becomes genuinely strong combined with mirrors nobody at LoopOver controls: GH Archive's hourly PushEvent export and Software Heritage's on-demand archival. A rewrite becomes independently detectable by checking an archive this repo doesn't control — documented as a runnable cross-mirror verification procedure in the module's own header comment.

  • Commits the identical canonicalized payload + signature Rekor anchors — the two backends commit to the same fact, never a reshaped copy.
  • Read-modify-write via the file's own sha as a compare-and-swap guard (GitHub 409s a stale-sha PUT).
  • Never throws past the caller: missing repo, auth failure, rate limit, a raced sha all record status: 'failed' via ledger: anchor persistence + public attempt log (success and failure both visible) #9271's persistence, matching the Rekor backend's identical posture.
  • Handles GitHub omitting inline content for files >1MB (falls back to empty existing content rather than crashing) — a real edge case, tested rather than suppressed.

Also fixes a real bug found while building this: ledger-anchor.ts's re-export statement only carried sha256Hex, silently dropping canonicalJson — any consumer importing it (this backend needs it to commit the same payload Rekor anchors) got undefined. Applied at the actual origin (#9270 / #9392) and rebased through the whole stack, not patched over downstream where it happened to surface.

Closes #9273.

Test plan

  • npx vitest run test/unit/ledger-anchor-git.test.ts — 8/8, 100% statement/branch/function/line coverage
  • octokit is always injected — never a real GitHub write; the property under test is this module's own append/error/persistence logic
  • First-anchor-ever (no prior sha), append-to-existing (sha used as compare-and-swap, content actually appended not overwritten), and the >1MB-file edge case (no inline content in the GET response) all proven, not assumed
  • Every failure path — rate limit/auth error, non-404 GET error, missing commit sha in the PUT response, generic exception — records status: 'failed' with the real error and resolves rather than rejects
  • npx tsc --noEmit clean · npx tsx scripts/check-import-specifiers.ts clean · npm run db:schema-drift:check clean

Base branch is feat/ledger-anchor-rekor (#9272) — merge order: #9392#9395#9397 → this.

@JSONbored JSONbored self-assigned this Jul 27, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb

loopover-orb Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-27 16:44:07 UTC

3 files · 1 AI reviewer · no blockers · readiness 77/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This adds a second, independent anchoring backend that appends a JSONL line to a GitHub-hosted file via the Contents API, reusing the injected-Octokit pattern and read-modify-write-with-sha compare-and-swap for concurrency safety. The logic is sound: 404 on GET is correctly treated as 'file doesn't exist yet', missing inline content (large-file case) falls back to empty rather than crashing, and every failure path is caught and persisted as status:'failed' rather than thrown. The bundled one-line fix to ledger-anchor.ts's re-export (restoring canonicalJson) is a real, narrowly-scoped bug fix directly needed by this backend, and the test suite exercises all the stated branches (create, append, large-file, 404-vs-other-error, missing-sha, generic catch) against a scripted requester.

Nits — 5 non-blocking
  • src/review/ledger-anchor-git.ts:71 uses the bare literal 404 to distinguish 'not found yet' from a real error; a named constant (e.g. HTTP_NOT_FOUND) would make the intent clearer at a glance.
  • src/review/ledger-anchor-git.ts:submitToGitAnchor nests an inner try/catch inside an outer try/catch plus an if — readable but at the edge of what's easy to follow; consider extracting the GET-with-404-fallback into a small helper.
  • The GET/PUT read-modify-write is not atomic beyond the sha compare-and-swap; a 409 from a raced writer is handled generically as a thrown error recorded as 'failed' rather than retried, which matches the stated posture but means a genuine race silently drops that anchor attempt without automatic retry (worth confirming the caller/job in ledger: scheduled checkpoint anchoring — hourly or every 256 records, whichever first #9274 retries on 'failed').
  • Consider adding a brief JSDoc note on submitToGitAnchor's exported type GitHubContentsRequester's `params` about which keys are required for PUT vs GET, since callers infer this only from usage.
  • If a scheduling job wraps this in retry logic, a comment or link to that job (ledger: scheduled checkpoint anchoring — hourly or every 256 records, whichever first #9274) here would help future readers understand what happens after a 'failed' status is recorded.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #9273
Related work ⚠️ 1 scoped overlap Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 13 registered-repo PR(s), 13 merged, 345 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 13 PR(s), 345 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff implements append-only Contents API commits with sha-based compare-and-swap, commits the identical canonicalized payload+signature used for Rekor, records status:'failed' via the persistence layer on any error without throwing, and documents a runnable GH Archive cross-mirror verification procedure in the module header, matching all four stated requirements.

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, Ruby, Go, MDX, Shell, Solidity, JavaScript
  • Official Gittensor activity: 13 PR(s), 345 issue(s).
  • Related work: Titles/paths share 4 meaningful terms. (PR #9397)
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Then work through the remaining 4 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@JSONbored
JSONbored force-pushed the feat/ledger-anchor-rekor branch from 1d00704 to bd01dfc Compare July 27, 2026 16:22
…hive (#9273)

Fifth sub-issue of #9267. Secondary, complementary to Rekor: appends one
JSONL line per anchor to a public repo via the GitHub Contents API and the
existing makeInstallationOctokit chokepoint -- never a direct fetch to the
GitHub API, matching every other GitHub write in this engine.

Alone this is weaker than Rekor (GitHub is a trusted third party, force-push
rewrites it). It becomes genuinely strong combined with mirrors nobody at
LoopOver controls: GH Archive's hourly PushEvent export and Software
Heritage's on-demand archival. A rewrite becomes independently detectable by
checking an archive this repo doesn't control -- documented as a runnable
cross-mirror verification procedure in the module's own header.

Commits the identical canonicalized payload + signature Rekor anchors, so
the two backends commit to the same fact, never a reshaped copy. Read-modify-
write via the file's own sha as a compare-and-swap guard; never throws past
the caller (missing repo, auth failure, rate limit, a raced sha all record
status:'failed' via #9271's persistence, same posture as the Rekor backend).

Also fixes a real bug found while building this: ledger-anchor.ts's re-export
statement only carried sha256Hex, dropping canonicalJson entirely -- any
consumer importing it (this backend needs it to commit the same payload Rekor
anchors) got undefined. Applied at the actual origin (#9270/PR #9392) and
rebased through the whole stack, not patched over downstream.
@JSONbored
JSONbored force-pushed the feat/ledger-anchor-git branch from 406f82d to 402d4c8 Compare July 27, 2026 16:23
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.60%. Comparing base (bd01dfc) to head (402d4c8).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@                    Coverage Diff                    @@
##           feat/ledger-anchor-rekor    #9403   +/-   ##
=========================================================
  Coverage                     88.60%   88.60%           
=========================================================
  Files                           841      842    +1     
  Lines                        109801   109828   +27     
  Branches                      26149    26153    +4     
=========================================================
+ Hits                          97288    97315   +27     
  Misses                        11541    11541           
  Partials                        972      972           
Flag Coverage Δ
backend 93.59% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/ledger-anchor-git.ts 100.00% <100.00%> (ø)
src/review/ledger-anchor.ts 100.00% <ø> (ø)

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 7f4eda6 into feat/ledger-anchor-rekor Jul 27, 2026
7 checks passed
@loopover-orb
loopover-orb Bot deleted the feat/ledger-anchor-git branch July 27, 2026 16:44
JSONbored added a commit that referenced this pull request Jul 27, 2026
…hive (#9273) (#9403)

Fifth sub-issue of #9267. Secondary, complementary to Rekor: appends one
JSONL line per anchor to a public repo via the GitHub Contents API and the
existing makeInstallationOctokit chokepoint -- never a direct fetch to the
GitHub API, matching every other GitHub write in this engine.

Alone this is weaker than Rekor (GitHub is a trusted third party, force-push
rewrites it). It becomes genuinely strong combined with mirrors nobody at
LoopOver controls: GH Archive's hourly PushEvent export and Software
Heritage's on-demand archival. A rewrite becomes independently detectable by
checking an archive this repo doesn't control -- documented as a runnable
cross-mirror verification procedure in the module's own header.

Commits the identical canonicalized payload + signature Rekor anchors, so
the two backends commit to the same fact, never a reshaped copy. Read-modify-
write via the file's own sha as a compare-and-swap guard; never throws past
the caller (missing repo, auth failure, rate limit, a raced sha all record
status:'failed' via #9271's persistence, same posture as the Rekor backend).

Also fixes a real bug found while building this: ledger-anchor.ts's re-export
statement only carried sha256Hex, dropping canonicalJson entirely -- any
consumer importing it (this backend needs it to commit the same payload Rekor
anchors) got undefined. Applied at the actual origin (#9270/PR #9392) and
rebased through the whole stack, not patched over downstream.
JSONbored added a commit that referenced this pull request Jul 27, 2026
* feat(ledger): Rekor v2 hashedrekord anchoring backend (#9272)

Fourth sub-issue of #9267. Primary anchoring mechanism per the mechanism
research: hashedrekord is exactly built for a plain hash + signature — Rekor
never sees the anchor payload itself. No Fulcio, no OIDC, no new dependency
(ECDSA P-256/SHA-256 is native WebCrypto, the same keypair #9270 already
produces). Shard URL is env-configurable with a documented fallback, never
hardcoded, per the research's explicit "shards roll annually" warning.

Every path — non-2xx response, an unparseable response body, a network
exception — records status:'failed' via #9271's persistence rather than
throwing past the caller, so #9273's git backend still gets attempted even if
this one fails.

Caught and fixed a real bug during testing: this module was pre-stringifying
its own error before handing it to recordLedgerAnchorAttempt, which then ran
errorMessage() a SECOND time on what was now a plain string — and errorMessage
only extracts .message from actual Error instances, so every failure path
collapsed to the generic "unknown error" fallback regardless of what actually
went wrong. Fixed by passing the raw unknown error through and teaching
persistence to handle both an already-built string and a genuine Error, so the
real diagnostic (a 429, a shape mismatch, "network down") reaches the public
listing instead of being silently discarded.

Storing the full Rekor TransparencyLogEntry (inclusion proof + signed
checkpoint) for fully offline verification is deliberately deferred — online
verification via rekor-cli against the stored shard URL + uuid works
completely without it today.

* feat(ledger): git-commit anchoring backend, cross-verified via GH Archive (#9273) (#9403)

Fifth sub-issue of #9267. Secondary, complementary to Rekor: appends one
JSONL line per anchor to a public repo via the GitHub Contents API and the
existing makeInstallationOctokit chokepoint -- never a direct fetch to the
GitHub API, matching every other GitHub write in this engine.

Alone this is weaker than Rekor (GitHub is a trusted third party, force-push
rewrites it). It becomes genuinely strong combined with mirrors nobody at
LoopOver controls: GH Archive's hourly PushEvent export and Software
Heritage's on-demand archival. A rewrite becomes independently detectable by
checking an archive this repo doesn't control -- documented as a runnable
cross-mirror verification procedure in the module's own header.

Commits the identical canonicalized payload + signature Rekor anchors, so
the two backends commit to the same fact, never a reshaped copy. Read-modify-
write via the file's own sha as a compare-and-swap guard; never throws past
the caller (missing repo, auth failure, rate limit, a raced sha all record
status:'failed' via #9271's persistence, same posture as the Rekor backend).

Also fixes a real bug found while building this: ledger-anchor.ts's re-export
statement only carried sha256Hex, dropping canonicalJson entirely -- any
consumer importing it (this backend needs it to commit the same payload Rekor
anchors) got undefined. Applied at the actual origin (#9270/PR #9392) and
rebased through the whole stack, not patched over downstream.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant