From ea364943c2b78f4a427a8edbe32f20f6cce504bc Mon Sep 17 00:00:00 2001 From: Yuval Date: Mon, 27 Jul 2026 10:29:38 +0300 Subject: [PATCH 1/3] feat(llms): agent instructions, per-page .md twins, apex llms.txt link MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three upgrades to the LLM-ingestion surface, following the patterns used by Stripe, ElevenLabs, Supabase, and Vercel: - llms.txt: add an 'Instructions for AI agents' section (Stripe/ElevenLabs pattern) — how to fetch a single page as Markdown, where the full corpus lives, a freshness warning for chain coverage, and where product/pricing facts live so agents don't infer them from these docs. - llms_full.py: emit a per-page Markdown twin next to each page's HTML (URL minus trailing slash + .md, e.g. /deployment/cache/ -> /deployment/cache.md; section indexes collapse to
.md; the root serves at /index.md). Lets an agent fetch one topic cheaply instead of the whole llms-full.txt corpus. Advertised in llms.txt and robots.txt. - Point the two www.magmadevs.com/llms.txt references at the apex https://magmadevs.com/llms.txt — the www URL 302-redirects, and naive crawlers that don't follow redirects get an empty body. Also: the Pages workflow now triggers on hooks/** and overrides/** changes (previously a hook edit alone never redeployed the site). Strict build passes; 36 per-page .md files verified in site/. Co-Authored-By: Claude Fable 5 --- .github/workflows/docs.yml | 2 ++ docs/llms.txt | 11 +++++++++-- docs/robots.txt | 2 ++ hooks/llms_full.py | 34 ++++++++++++++++++++++++++++++---- mkdocs.yml | 5 +++-- 5 files changed, 46 insertions(+), 8 deletions(-) diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 50f3f89..93fa9bc 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -9,6 +9,8 @@ on: - main paths: - docs/** + - hooks/** + - overrides/** - mkdocs.yml - requirements.txt - .github/workflows/docs.yml diff --git a/docs/llms.txt b/docs/llms.txt index 67abdc3..2e6aa2b 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -1,11 +1,18 @@ # Smart Router — Documentation -> Developer documentation for Smart Router, the RPC orchestration and security layer by Magma Devs. These docs cover the self-hosted, open-source core: running it, connecting your app, configuring QoS routing, caching, and the failover and cross-validation pipeline, and operating it across 75+ chain specs. For product, pricing, security, and company details, see https://www.magmadevs.com/llms.txt. The full text of every page in this documentation is available as a single file at https://docs.magmadevs.com/llms-full.txt. +> Developer documentation for Smart Router, the RPC orchestration and security layer by Magma Devs. These docs cover the self-hosted, open-source core: running it, connecting your app, configuring QoS routing, caching, and the failover and cross-validation pipeline, and operating it across 75+ chain specs. For product, pricing, security, and company details, see https://magmadevs.com/llms.txt. The full text of every page in this documentation is available as a single file at https://docs.magmadevs.com/llms-full.txt. Smart Router sits between your application and a fleet of RPC providers (Alchemy, Infura, QuickNode, Helius, self-hosted nodes, and others). It picks a healthy, performant provider per request, retries the bad one, hedges the slow one, validates responses across providers, and caches what it can — so no single provider's uptime or data integrity becomes a point of failure. Think of it as Cloudflare for blockchain RPC. These docs are for the open-source core (PolyForm Noncommercial license); managed Pro and Enterprise tiers are described on the main site. Smart Router speaks JSON-RPC, REST, gRPC, and Tendermint RPC; the default local endpoint is `http://127.0.0.1:3360`. Chains are described by JSON specs — 75 specs spanning 67 mainnets and 64 testnets (100+ networks) across EVM, Cosmos/Tendermint, Solana, and more. +## Instructions for AI agents + +- For clean Markdown of a single page, drop the URL's trailing slash and append `.md` — e.g. https://docs.magmadevs.com/deployment/cache/ → https://docs.magmadevs.com/deployment/cache.md. The quick start (site root) is at https://docs.magmadevs.com/index.md. +- The full text of every page in one file: https://docs.magmadevs.com/llms-full.txt. Prefer single-page `.md` fetches when you only need one topic. +- Chain coverage, spec counts, and configuration options change frequently. Cite the live pages (especially https://docs.magmadevs.com/reference/chains/) rather than relying on trained knowledge. +- Config examples in these docs are YAML for the self-hosted open-source core. For product, pricing, security, and company facts, use https://magmadevs.com/llms.txt — do not infer pricing or tier limits from these docs. + ## Getting started - [Quick start](https://docs.magmadevs.com/): Get Smart Router running in three commands — clone the repo, start it with Docker Compose, and send your first JSON-RPC relay. @@ -63,7 +70,7 @@ These docs are for the open-source core (PolyForm Noncommercial license); manage ## Resources - [Magma Devs (main site)](https://www.magmadevs.com): Product, positioning, customers, and company. -- [Magma Devs llms.txt](https://www.magmadevs.com/llms.txt): Product, pricing, security, and citation facts for Smart Router. +- [Magma Devs llms.txt](https://magmadevs.com/llms.txt): Product, pricing, security, and citation facts for Smart Router. - [Smart Router (product)](https://www.magmadevs.com/product/smart-router): How the orchestration layer works. - [Pricing](https://www.magmadevs.com/pricing): Open Source, Pro, and Enterprise tiers. - [GitHub: Magma-Devs/smart-router](https://github.com/Magma-Devs/smart-router): The open-source core — source, issues, and releases. diff --git a/docs/robots.txt b/docs/robots.txt index 3449794..e75e966 100644 --- a/docs/robots.txt +++ b/docs/robots.txt @@ -10,3 +10,5 @@ Sitemap: https://docs.magmadevs.com/sitemap.xml # LLM-friendly index: https://docs.magmadevs.com/llms.txt # LLM full-text corpus: https://docs.magmadevs.com/llms-full.txt +# Per-page Markdown: drop a page URL's trailing slash and append .md +# (e.g. /deployment/cache/ -> /deployment/cache.md) diff --git a/hooks/llms_full.py b/hooks/llms_full.py index 8cf494f..79ba1e7 100644 --- a/hooks/llms_full.py +++ b/hooks/llms_full.py @@ -1,5 +1,5 @@ """ -MkDocs hook: generate `llms-full.txt` at build time. +MkDocs hook: generate `llms-full.txt` and per-page `.md` twins at build time. `llms.txt` (hand-maintained in docs/) is the *index* — a curated map of the docs with one-line descriptions. `llms-full.txt` is the *corpus* — the full @@ -7,9 +7,15 @@ crawlers (ChatGPT, Claude, Perplexity) ingest to answer questions about Smart Router accurately. -We derive it from the rendered page set so it never drifts from the real docs: -on `on_page_markdown` we capture each page's source Markdown; on -`on_post_build` we write them out in nav order to site/llms-full.txt. +Each page is also emitted as a standalone Markdown file next to its HTML +(the Stripe/Supabase/ElevenLabs convention): drop the page URL's trailing +slash and append `.md` — /deployment/cache/ → /deployment/cache.md, section +indexes like /deployment/ → /deployment.md, and the root page → /index.md. +This lets an agent fetch one page cheaply instead of the whole corpus. + +We derive everything from the rendered page set so it never drifts from the +real docs: on `on_page_markdown` we capture each page's source Markdown; on +`on_post_build` we write site/llms-full.txt and the per-page twins. Wired via `hooks:` in mkdocs.yml. No manual upkeep. """ @@ -96,6 +102,26 @@ def on_post_build(*, config): with open(out_path, "w", encoding="utf-8") as fh: fh.write("\n".join(lines).rstrip() + "\n") + # Per-page Markdown twins. src_uri "deployment/cache.md" already matches + # its public path; section indexes collapse ("deployment/index.md" → + # "deployment.md") so the URL rule is uniformly "trailing slash → .md". + # The root "index.md" stays as-is and serves at /index.md. + for uri in ordered: + title, md = _PAGES[uri] + if uri != "index.md" and uri.endswith("/index.md"): + dest_uri = uri[: -len("/index.md")] + ".md" + else: + dest_uri = uri + page_url = f"{site_url}/{uri[:-3].rstrip('/')}/" if uri.endswith(".md") else f"{site_url}/{uri}" + page_url = page_url.replace("/index/", "/") + dest_path = os.path.join(site_dir, *dest_uri.split("/")) + os.makedirs(os.path.dirname(dest_path) or site_dir, exist_ok=True) + text = md.strip() + # Only synthesize a title when the page body doesn't open with its own H1. + heading = "" if text.startswith("# ") else f"# {title}\n\n" + with open(dest_path, "w", encoding="utf-8") as fh: + fh.write(f"{heading}Source: {page_url}\n\n{text}\n") + # MkDocs skips dot-directories in docs/, so docs/.well-known/ is not copied # to the build. Copy it through explicitly so /.well-known/security.txt # (RFC 9116) is served. diff --git a/mkdocs.yml b/mkdocs.yml index a181e0f..878b3ba 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -56,8 +56,9 @@ plugins: enable_creation_date: false # Build hooks. llms_full.py concatenates every page into site/llms-full.txt -# (the LLM-ingestion corpus) in nav order at post-build — always in sync with -# content, no manual upkeep. The curated index is the hand-written docs/llms.txt. +# (the LLM-ingestion corpus) in nav order at post-build, and emits a per-page +# Markdown twin next to each page (URL minus trailing slash + .md) — always in +# sync with content, no manual upkeep. The curated index is docs/llms.txt. hooks: - hooks/llms_full.py From 1d0f2d6931eee37c93b26a18701177db20390af2 Mon Sep 17 00:00:00 2001 From: Yuval Date: Mon, 27 Jul 2026 10:58:14 +0300 Subject: [PATCH 2/3] =?UTF-8?q?feat(llms):=20'Copy=20page'=20button=20?= =?UTF-8?q?=E2=80=94=20copy=20any=20page=20as=20Markdown=20for=20LLMs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Mintlify-style affordance next to every page H1: - Main button fetches the page's .md twin (same-origin) and copies it to the clipboard, flipping to 'Copied' for 2s. Falls back to the legacy execCommand copy where the async Clipboard API is denied, and to opening the raw .md in a new tab if copying is impossible. - Dropdown: View as Markdown, Open in ChatGPT, Open in Claude — share links use the canonical public URL so the assistant can fetch the page. - Styling follows the existing mdx-* conventions in extra.css; collapses to full-width above the H1 on mobile. Verified in the built site: twin fetch (200, 4.7KB), copied payload, success state, fallback path, and menu URLs. Co-Authored-By: Claude Fable 5 --- docs/javascripts/copy-page.js | 98 +++++++++++++++++++++++++++++++++++ docs/stylesheets/extra.css | 73 ++++++++++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 172 insertions(+) create mode 100644 docs/javascripts/copy-page.js diff --git a/docs/javascripts/copy-page.js b/docs/javascripts/copy-page.js new file mode 100644 index 0000000..fc67202 --- /dev/null +++ b/docs/javascripts/copy-page.js @@ -0,0 +1,98 @@ +// "Copy page" button (Mintlify-style) injected next to each page's H1. +// Copies the page's Markdown twin (URL minus trailing slash + .md — emitted by +// hooks/llms_full.py) so readers can paste a clean page into an LLM, plus a +// dropdown to view the raw Markdown or open the page in ChatGPT / Claude. +// Re-runs on every instant-nav load; no-ops if the button is already present. +document$.subscribe(function () { + var article = document.querySelector(".md-content article"); + var h1 = article && article.querySelector("h1"); + if (!article || !h1) return; + if (article.querySelector(".mdx-copy-page")) return; // already added + + // Directory-style URL -> Markdown twin: trailing slash becomes .md; + // the site root serves at /index.md. + var mdPath = location.pathname.replace(/\/+$/, ""); + mdPath = mdPath === "" ? "/index.md" : mdPath + ".md"; + // Clipboard copy fetches same-origin (works on any host, incl. local + // builds); the share links use the canonical public URL so ChatGPT/Claude + // can actually reach the page. + var fetchUrl = location.origin + mdPath; + var canonical = document.querySelector('link[rel="canonical"]'); + var mdUrl = canonical ? new URL(mdPath, canonical.href).href : fetchUrl; + + var askPrompt = encodeURIComponent("Read " + mdUrl + " so I can ask questions about this documentation page."); + + var wrap = document.createElement("div"); + wrap.className = "mdx-copy-page"; + wrap.innerHTML = + '" + + '" + + '"; + + h1.parentNode.insertBefore(wrap, h1); + + var btn = wrap.querySelector(".mdx-copy-page__btn"); + var label = btn.querySelector("span"); + var caret = wrap.querySelector(".mdx-copy-page__caret"); + var menu = wrap.querySelector(".mdx-copy-page__menu"); + + // Async Clipboard API first; hidden-textarea execCommand fallback for + // contexts where the clipboard-write permission is denied. + function copyText(text) { + return navigator.clipboard.writeText(text).catch(function () { + var ta = document.createElement("textarea"); + ta.value = text; + ta.setAttribute("readonly", ""); + ta.style.position = "fixed"; + ta.style.opacity = "0"; + document.body.appendChild(ta); + ta.select(); + var ok = document.execCommand("copy"); + document.body.removeChild(ta); + if (!ok) throw new Error("copy failed"); + }); + } + + btn.addEventListener("click", function () { + fetch(fetchUrl) + .then(function (r) { + if (!r.ok) throw new Error(r.status); + return r.text(); + }) + .then(copyText) + .then(function () { + label.textContent = "Copied"; + btn.classList.add("mdx-copy-page__btn--done"); + setTimeout(function () { + label.textContent = "Copy page"; + btn.classList.remove("mdx-copy-page__btn--done"); + }, 2000); + }) + .catch(function () { + // Clipboard blocked or twin missing — fall back to showing the raw file. + window.open(mdUrl, "_blank", "noopener"); + }); + }); + + caret.addEventListener("click", function (e) { + e.stopPropagation(); + var open = !menu.hidden; + menu.hidden = open; + caret.setAttribute("aria-expanded", String(!open)); + }); + document.addEventListener("click", function () { + if (!menu.hidden) { + menu.hidden = true; + caret.setAttribute("aria-expanded", "false"); + } + }); +}); diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 22a0de0..c2ed610 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -2013,3 +2013,76 @@ html { scroll-padding-top: 96px; } font-size: 0.85rem; color: var(--text-muted, rgb(var(--gray-500))); } + +/* ============================================================ + "Copy page" button (javascripts/copy-page.js) — Mintlify-style + LLM affordance floated to the right of each page's H1. + ============================================================ */ +.mdx-copy-page { + position: relative; + float: right; + display: inline-flex; + align-items: stretch; + margin: 0.4rem 0 0.5rem 1rem; + font-size: 0.62rem; + line-height: 1; +} +.mdx-copy-page__btn, +.mdx-copy-page__caret { + display: inline-flex; + align-items: center; + gap: 0.3rem; + padding: 0.38rem 0.55rem; + border: 1px solid var(--md-default-fg-color--lightest, rgb(var(--gray-200))); + background: var(--surface, #fff); + color: var(--text-body, rgb(var(--gray-700))); + cursor: pointer; + font: inherit; + font-weight: 500; +} +.mdx-copy-page__btn { + border-radius: 0.35rem 0 0 0.35rem; +} +.mdx-copy-page__caret { + border-left: none; + border-radius: 0 0.35rem 0.35rem 0; + padding: 0.38rem 0.3rem; +} +.mdx-copy-page__btn:hover, +.mdx-copy-page__caret:hover { + border-color: var(--primary); + color: var(--primary); +} +.mdx-copy-page__btn--done, +.mdx-copy-page__btn--done:hover { + border-color: var(--primary); + color: var(--primary); +} +.mdx-copy-page__menu { + position: absolute; + top: calc(100% + 0.25rem); + right: 0; + z-index: 10; + min-width: 10.5rem; + padding: 0.25rem; + border: 1px solid var(--md-default-fg-color--lightest, rgb(var(--gray-200))); + border-radius: 0.4rem; + background: var(--surface, #fff); + box-shadow: 0 4px 16px rgba(0, 0, 0, 0.08); +} +.mdx-copy-page__menu a { + display: block; + padding: 0.45rem 0.6rem; + border-radius: 0.3rem; + color: var(--text-body, rgb(var(--gray-700))); +} +.mdx-copy-page__menu a:hover { + background: var(--md-default-bg-color--light, rgb(var(--gray-50))); + color: var(--primary); +} +@media screen and (max-width: 44.9375em) { + .mdx-copy-page { + float: none; + margin: 0 0 0.75rem; + } +} diff --git a/mkdocs.yml b/mkdocs.yml index 878b3ba..7bc7ef3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -115,6 +115,7 @@ extra_javascript: - javascripts/consent-mode.js - javascripts/tablesort.js - javascripts/toc-title.js + - javascripts/copy-page.js - javascripts/search-hint.js - javascripts/nav-top.js - javascripts/mobile-nav-search.js From d702e08612da7ea163c66b6dff817193a19793da Mon Sep 17 00:00:00 2001 From: Yuval Date: Mon, 27 Jul 2026 11:34:17 +0300 Subject: [PATCH 3/3] fix(llms): copy the assistant prompt on Open in ChatGPT/Claude clicks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ChatGPT and Claude drop the ?q= prefill whenever they bounce the visitor through login or a session refresh — there is nothing the link itself can do about it. Safety net: clicking either assistant link also copies the prompt ('Read .md so I can ask questions...') to the clipboard so the visitor can paste it when the prefill is lost. Verified across all 36 pages of the built site: button injected, both assistant links carry identical well-formed prompts, every referenced .md twin resolves, and the click-to-copy fires on both links. Co-Authored-By: Claude Fable 5 --- docs/javascripts/copy-page.js | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/javascripts/copy-page.js b/docs/javascripts/copy-page.js index fc67202..c279c3e 100644 --- a/docs/javascripts/copy-page.js +++ b/docs/javascripts/copy-page.js @@ -83,6 +83,15 @@ document$.subscribe(function () { }); }); + // Safety net: ChatGPT/Claude drop the ?q= prompt when they bounce the + // visitor through login. Copy the prompt on click so it can be pasted. + var promptText = "Read " + mdUrl + " so I can ask questions about this documentation page."; + menu.querySelectorAll('a[href*="chatgpt.com"], a[href*="claude.ai"]').forEach(function (a) { + a.addEventListener("click", function () { + copyText(promptText).catch(function () {}); + }); + }); + caret.addEventListener("click", function (e) { e.stopPropagation(); var open = !menu.hidden;