feat(orb): serve GET /v1/public/eval-scores — the v1 EvalScoreRecord transport#9283
Open
JSONbored wants to merge 2 commits into
Open
feat(orb): serve GET /v1/public/eval-scores — the v1 EvalScoreRecord transport#9283JSONbored wants to merge 2 commits into
JSONbored wants to merge 2 commits into
Conversation
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| 🔵 In progress View logs |
loopover-ui | b90f7e5 | Jul 27 2026, 11:29 AM |
…transport (#9266) Reshapes the already-computed outcome-confirmed rule precision into the EvalScoreRecord shape spec'd on #9215: per-record digest commitment, corpus-checksum-anchored, independently re-derivable by any consumer without trusting the transport. Same flag/cache posture as /v1/public/stats since this is a reshaping of the same data, not a new scoring surface. Extends PublicRulePrecisionRow with a raw confirmed count (previously only the rounded precision ratio was exposed) so EvalScoreRecord.score.confirmed is exact rather than reconstructed from an already-rounded value.
JSONbored
force-pushed
the
feat/eval-scores-endpoint
branch
from
July 27, 2026 11:29
a897e90 to
b90f7e5
Compare
…onalPropertyTypes
Bundle ReportChanges will increase total bundle size by 2.12kB (0.03%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: loopover-uiAssets Changed:
|
loopover-orb
Bot
added
the
gittensor:bug
❌ 5 Tests Failed:
View the top 3 failed test(s) by shortest run time
View the full list of 1 ❄️ flaky test(s)
To view more test analytics, go to the Test Analytics Dashboard |
Contributor
|
Caution 🛑 LoopOver review result - fixes requiredReview updated: 2026-07-27 11:46:22 UTC
Review summary Nits — 4 non-blocking
CI checks failing
Decision drivers
Context & advisory signals — never blocks the verdict
Linked issue satisfactionPartially addressed Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Decision record
Visual preview
Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy · Diff highlights exactly what changed. Scroll preview
A short scroll-through clip (desktop) — click either thumbnail to open the full animation. Evidence for scroll-linked behavior a single screenshot can't show. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GET /v1/public/eval-scores?subject=&since=— the v1 transport Spec: validator-facing eval interface — what SN74 consumes from LoopOver as the objective eval provider #9215 specs forEvalScoreRecords. Unauthenticated, sameLOOPOVER_PUBLIC_STATSflag/cache posture as/v1/public/stats.outcome_confirmed_precisionsource now: reshapesloadPublicRulePrecision's already-computed data into per-record, digest-committedEvalScoreRecords, one per rule, anchored to the latest persisted backtest run's corpus checksum. Emits nothing when there's no freeze point to commit to yet (a record whose commitments can't be independently re-derived isn't publishable, per Spec: validator-facing eval interface — what SN74 consumes from LoopOver as the objective eval provider #9215).PublicRulePrecisionRowgains a rawconfirmedcount (previously only the roundedprecisionratio was exposed) — needed soEvalScoreRecord.score.confirmedis exact, not reconstructed by inverting an already-rounded value. This also flows through/v1/public/stats(which spreads the block verbatim); OpenAPI schema updated to match.recall/abstaineddon't apply to this work-unit kind (ORB's rules fire deterministically, no agent abstention concept here) —recall: null(genuinely inapplicable),abstained: 0(correctly zero, not a masked null).One deliberate deviation from #9266's own written requirement: the issue asked for "404 disabled / 503 on internal error." I didn't add a try/catch —
loadPublicRulePrecisionis fail-safe internally (safeAllswallows every read error into an empty section, by its own doc comment) and the rest of the pipeline is pure, so there's no reachable throw path in this composition today, and/v1/public/statsitself doesn't test its own 503 path for the identical reason. Added a comment explaining this and left real error handling for when #9265's benchmark-run source (which will have real IO) actually needs it.Closes #9266.
Test plan
npx vitest run test/unit/eval-score-records.test.ts— 12/12 passing, 100% statement/branch/function/line coverage on the new core modulenpx vitest run test/unit/public-rule-precision.test.ts— updated for the newconfirmedfield, 5/5 passingnpx vitest run test/integration/public-eval-scores-route.test.ts— 7/7 passing: flag-gated 404, empty-array on no freeze point, digest recomputability, subject/since filtering, cache headers, fail-safe degradation on a broken storenpx vitest run test/integration/public-decision-ledger-routes.test.ts— added to the shared exempt-public-routes table (14/14 passing)npx vitest run test/unit/openapi.test.ts— 3/3 passingnpm run ui:openapi/ui:openapi:check— regenerated and verified cleannpx tsx scripts/check-import-specifiers.ts— clean