feat(console): route Indexer and search through CONSOLE_DATA_API_URL - #136
Conversation
Cut board and RustyWeb search over to the consumer GraphQL door so private MCP/node tokens stay off those paths (HANDOFF-CONSOLE-SINGLE-DOOR-1.0). Co-authored-by: Cursor <cursoragent@cursor.com>
|
ECC bundle files are already tracked in this repository. Skipping generation of another bundle PR. |
📝 WalkthroughWalkthroughConsole GraphQL routing now prioritizes the data API door. Indexer object reads and RustyWeb searches use authenticated, timed consumer GraphQL requests, with updated error handling, tests, and single-door documentation. ChangesConsole single-door GraphQL routing
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR routes the Console’s Indexer object reads and RustyWeb live search through the CommonPlace consumer GraphQL door (CONSOLE_DATA_API_URL), while documenting the “single door” cutover and leaving explicit MCP-only exceptions in place.
Changes:
- Switch Indexer board object loading to use consumer GraphQL over
CONSOLE_DATA_API_URL(with upstream credential headers + timeouts). - Switch web research (RustyWeb search) to call the consumer GraphQL
rustyWebSearchfield overCONSOLE_DATA_API_URL. - Add/adjust tests and record the cutover in an ADR-style doc.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| docs/records/011-console-single-door.md | Adds a record describing the single-door rule, cutover evidence, and remaining MCP exceptions. |
| apps/console/src/lib/server/web-research.ts | Replaces the Node /v1/rustyweb/search call with a consumer GraphQL rustyWebSearch call. |
| apps/console/src/lib/server/indexer-harness.ts | Replaces MCP GraphQL transport for topicIndexerObjects with consumer GraphQL transport; keeps preview assets on MCP. |
| apps/console/src/lib/server/indexer-harness.test.ts | Updates Indexer transport tests for the new fetch-based consumer GraphQL path. |
| apps/console/src/lib/server/consumer-graphql.ts | Updates endpoint selection to prefer CONSOLE_DATA_API_URL. |
| apps/console/src/lib/server/consumer-graphql.test.ts | Adds coverage for preferring CONSOLE_DATA_API_URL when both env vars exist. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const configured = | ||
| environment.CONSOLE_DATA_API_URL?.trim() | ||
| || environment.THEOREM_GRAPHQL_URL?.trim(); | ||
| if (!configured) return null; | ||
|
|
||
| const base = configured.replace(/\/+$/, ''); |
| if (!upstream.ok || payload?.errors || !payload?.data) { | ||
| const detail = payload?.errors?.[0]?.message; | ||
| return { | ||
| ok: false, | ||
| status: upstream.ok ? 502 : upstream.status, | ||
| error: typeof detail === 'string' ? detail : indexerTransportError(upstream.status, timeout.didTimeout()), | ||
| }; | ||
| } |
| function indexerTransportError(status: number, timedOut: boolean): string { | ||
| if (timedOut) return 'indexer_graphql_timeout'; | ||
| if (status === 404) return 'indexer_graphql_unconfigured'; | ||
| return 'indexer_graphql_failed'; | ||
| } |
| // Preview assets remain on the agent GraphQL surface until commonplace-api | ||
| // mounts topicPreviewAsset; Indexer object reads already use the data door. | ||
| const { callHarnessGraphql } = await import('@/lib/server/harness-graphql'); | ||
| const result = await callHarnessGraphql( |
| it('maps transport failures to the Indexer vocabulary', async () => { | ||
| fetchMock.mockResolvedValue({ | ||
| ok: false, | ||
| status: 504, | ||
| error: 'harness_graphql_timeout', | ||
| json: async () => ({ errors: [{ message: 'timeout' }] }), | ||
| }); |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 70e4108b50
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const configured = | ||
| environment.CONSOLE_DATA_API_URL?.trim() | ||
| || environment.THEOREM_GRAPHQL_URL?.trim(); |
There was a problem hiding this comment.
When CONSOLE_DATA_API_URL is configured, this redirects every consumerGraphqlUrl() caller to commonplace-api, but proactivity-harness.ts and filing-harness.ts still authenticate exclusively with THEOREM_API_KEY rather than CONSOLE_DATA_API_KEY. In the documented Railway configuration where the data API has its own key, both existing surfaces will receive 403 responses; either scope the new endpoint selection to Indexer/search or migrate those callers' credentials simultaneously.
AGENTS.md reference: apps/console/AGENTS.md:L33-L36
Useful? React with 👍 / 👎.
| readonly includeCaptures?: boolean; | ||
| }): Promise<IndexerRead> { | ||
| const result = await callHarnessGraphql(INDEXER_OBJECTS_QUERY, { | ||
| const result = await executeConsumerGraphql(INDEXER_OBJECTS_QUERY, { |
There was a problem hiding this comment.
This switches Indexer reads to the consumer schema, but a repo-wide search of this commit's apps/commonplace-api implementation finds no topicIndexerObjects field; the Query implementation beginning at apps/commonplace-api/src/schema/mod.rs:1808 does not declare it. Consequently commonplace-api returns a GraphQL unknown-field error for every Indexer object read, whereas the previous MCP schema owned this field.
AGENTS.md reference: AGENTS.md:L27-L27
Useful? React with 👍 / 👎.
| query ConsoleRustyWebSearch($query: String!, $limit: Int, $providers: [String!]) { | ||
| rustyWebSearch(query: $query, limit: $limit, providers: $providers) | ||
| } |
There was a problem hiding this comment.
The new request queries rustyWebSearch, but a repo-wide search of this commit's apps/commonplace-api schema and source finds no such field or resolver. Thus both Composer web research and Indexer live search receive an unknown-field GraphQL error from the configured data API and return 502 instead of results; the consumer field needs to land with this transport cutover.
Useful? React with 👍 / 👎.
| } finally { | ||
| timeout.clear(); | ||
| } | ||
|
|
||
| if (!upstream.ok) { | ||
| const envelope = await upstream.json().catch(() => null) as { |
There was a problem hiding this comment.
When the data API sends response headers but delays or stalls its JSON body, fetch() resolves and this finally clears the abort timer before upstream.json() starts on line 117. That leaves both Composer and Indexer search requests able to hang indefinitely despite the configured timeout; parse the body inside the timed try and clear the timer only after parsing completes.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
apps/console/src/lib/server/indexer-harness.test.ts (1)
91-102: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTest doesn't actually exercise
indexerTransportError's vocabulary mapping.This test's asserted
error: 'timeout'comes from the raw GraphQLerrors[0].messagepassthrough, not fromindexerTransportError. None of that function's three branches (timeout / 404-unconfigured / generic-failed) are covered by a case without a stringerrors[0].message. Consider adding a case with a non-string/absent error detail to confirm the normalized vocabulary is actually returned.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/console/src/lib/server/indexer-harness.test.ts` around lines 91 - 102, The test for readIndexerObjects currently validates raw GraphQL error-message passthrough rather than indexerTransportError's vocabulary mapping. Update or add cases that omit errors[0].message or provide a non-string detail, covering the timeout, 404-unconfigured, and generic-failed branches, and assert each normalized vocabulary value.apps/console/src/lib/server/indexer-harness.ts (1)
53-111: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsumer-GraphQL transport logic duplicated across two files.
executeConsumerGraphqlandloadWebResearchindependently implement the same endpoint-check, credential-resolution, timed-fetch-with-headers, and envelope-parsing sequence; the shared root cause is the lack of a common helper inconsumer-graphql.tsfor this transport shape.
apps/console/src/lib/server/indexer-harness.ts#L53-L111: extract the endpoint/credential/timeout/fetch/envelope logic into a shared helper (e.g. inconsumer-graphql.ts) and haveexecuteConsumerGraphqldelegate to it, keeping only the Indexer-specific principal resolution and result shape here.apps/console/src/lib/server/web-research.ts#L51-L116: adopt the same shared helper for the fetch/timeout/envelope portion, keeping only the RustyWeb-specific query/variables and source extraction here.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/console/src/lib/server/indexer-harness.ts` around lines 53 - 111, The Consumer-GraphQL transport sequence is duplicated and should be centralized. In apps/console/src/lib/server/indexer-harness.ts#L53-L111, extract endpoint validation, credential resolution, timeout-managed fetch, and envelope parsing from executeConsumerGraphql into a shared helper in consumer-graphql.ts, leaving principal resolution and the Indexer-specific result shape local. In apps/console/src/lib/server/web-research.ts#L51-L116, replace the duplicated transport logic in loadWebResearch with that helper while retaining its RustyWeb query/variables and source extraction.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/console/src/lib/server/indexer-harness.ts`:
- Around line 93-99: Stop returning raw upstream GraphQL messages from
executeConsumerGraphql and loadWebResearch. In
apps/console/src/lib/server/indexer-harness.ts lines 93-99, log the extracted
detail server-side and always return the fixed indexerTransportError(...)
result; in apps/console/src/lib/server/web-research.ts lines 122-134, log the
upstream detail and return the existing generic refusal message instead.
---
Nitpick comments:
In `@apps/console/src/lib/server/indexer-harness.test.ts`:
- Around line 91-102: The test for readIndexerObjects currently validates raw
GraphQL error-message passthrough rather than indexerTransportError's vocabulary
mapping. Update or add cases that omit errors[0].message or provide a non-string
detail, covering the timeout, 404-unconfigured, and generic-failed branches, and
assert each normalized vocabulary value.
In `@apps/console/src/lib/server/indexer-harness.ts`:
- Around line 53-111: The Consumer-GraphQL transport sequence is duplicated and
should be centralized. In
apps/console/src/lib/server/indexer-harness.ts#L53-L111, extract endpoint
validation, credential resolution, timeout-managed fetch, and envelope parsing
from executeConsumerGraphql into a shared helper in consumer-graphql.ts, leaving
principal resolution and the Indexer-specific result shape local. In
apps/console/src/lib/server/web-research.ts#L51-L116, replace the duplicated
transport logic in loadWebResearch with that helper while retaining its RustyWeb
query/variables and source extraction.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: cb7a7761-9315-4658-bd6d-ddf3249f035e
📒 Files selected for processing (6)
apps/console/src/lib/server/consumer-graphql.test.tsapps/console/src/lib/server/consumer-graphql.tsapps/console/src/lib/server/indexer-harness.test.tsapps/console/src/lib/server/indexer-harness.tsapps/console/src/lib/server/web-research.tsdocs/records/011-console-single-door.md
| if (!upstream.ok || payload?.errors || !payload?.data) { | ||
| const detail = payload?.errors?.[0]?.message; | ||
| return { | ||
| ok: false, | ||
| status: upstream.ok ? 502 : upstream.status, | ||
| error: typeof detail === 'string' ? detail : indexerTransportError(upstream.status, timeout.didTimeout()), | ||
| }; |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Files:"
git ls-files | rg 'apps/console/src/lib/server/(indexer-harness|web-research)\.ts$' || true
echo
echo "indexer-harness outline:"
ast-grep outline apps/console/src/lib/server/indexer-harness.ts --view expanded || true
echo
echo "web-research outline:"
ast-grep outline apps/console/src/lib/server/web-research.ts --view expanded || true
echo
echo "Relevant slices:"
wc -l apps/console/src/lib//.
sed -n '1,140p' apps/console/src/lib/server/indexer-harness.ts
echo '--- web-research 1-170 ---'
sed -n '1,170p' apps/console/src/lib/server/web-research.tsRepository: Travis-Gilbert/CommonPlace
Length of output: 1511
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo '--- indexer-harness 93-130, 174-210 ---'
sed -n '93,130p' apps/console/src/lib/server/indexer-harness.ts
sed -n '174,210p' apps/console/src/lib/server/indexer-harness.ts
echo
echo '--- web-research 120-145 ---'
sed -n '120,145p' apps/console/src/lib/server/web-research.ts
echo
echo '--- GraphQL error message fields in consumer-graphql helpers ---'
rg -n "errors\\?\\.\\[0\\]\\.message|envelope\\?.*errors|GRAPHQL|THEOREM|CONSOLE_HARNESS|graphql_query" apps/console/src/lib/server apps/console/src -g '*.ts' -g '*.tsx' | head -200Repository: Travis-Gilbert/CommonPlace
Length of output: 19733
Do not expose raw upstream GraphQL error messages. Both executeConsumerGraphql and loadWebResearch return errors[0].message directly to clients when it is a string; log that detail server-side and return fixed local error text/codes (indexerTransportError(...) for the indexer path and the existing generic refusal message for web research) instead.
📍 Affects 2 files
apps/console/src/lib/server/indexer-harness.ts#L93-L99(this comment)apps/console/src/lib/server/web-research.ts#L122-L134
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@apps/console/src/lib/server/indexer-harness.ts` around lines 93 - 99, Stop
returning raw upstream GraphQL messages from executeConsumerGraphql and
loadWebResearch. In apps/console/src/lib/server/indexer-harness.ts lines 93-99,
log the extracted detail server-side and always return the fixed
indexerTransportError(...) result; in
apps/console/src/lib/server/web-research.ts lines 122-134, log the upstream
detail and return the existing generic refusal message instead.
Summary
CONSOLE_DATA_API_URL).docs/records/011-console-single-door.md(HANDOFF-CONSOLE-SINGLE-DOOR-1.0).Test plan
vitestforconsumer-graphql+indexer-harnessCONSOLE_HARNESS_*on that pathrustyWebSearchSummary by CodeRabbit
New Features
Bug Fixes
Documentation