LLM ingestion: agent instructions, per-page .md pages, apex llms.txt link - #26
Open
yuvalava wants to merge 3 commits into
Open
LLM ingestion: agent instructions, per-page .md pages, apex llms.txt link#26yuvalava wants to merge 3 commits into
yuvalava wants to merge 3 commits into
Conversation
Three upgrades to the LLM-ingestion surface, following the patterns used by Stripe, ElevenLabs, Supabase, and Vercel: - llms.txt: add an 'Instructions for AI agents' section (Stripe/ElevenLabs pattern) — how to fetch a single page as Markdown, where the full corpus lives, a freshness warning for chain coverage, and where product/pricing facts live so agents don't infer them from these docs. - llms_full.py: emit a per-page Markdown twin next to each page's HTML (URL minus trailing slash + .md, e.g. /deployment/cache/ -> /deployment/cache.md; section indexes collapse to <section>.md; the root serves at /index.md). Lets an agent fetch one topic cheaply instead of the whole llms-full.txt corpus. Advertised in llms.txt and robots.txt. - Point the two www.magmadevs.com/llms.txt references at the apex https://magmadevs.com/llms.txt — the www URL 302-redirects, and naive crawlers that don't follow redirects get an empty body. Also: the Pages workflow now triggers on hooks/** and overrides/** changes (previously a hook edit alone never redeployed the site). Strict build passes; 36 per-page .md files verified in site/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mintlify-style affordance next to every page H1: - Main button fetches the page's .md twin (same-origin) and copies it to the clipboard, flipping to 'Copied' for 2s. Falls back to the legacy execCommand copy where the async Clipboard API is denied, and to opening the raw .md in a new tab if copying is impossible. - Dropdown: View as Markdown, Open in ChatGPT, Open in Claude — share links use the canonical public URL so the assistant can fetch the page. - Styling follows the existing mdx-* conventions in extra.css; collapses to full-width above the H1 on mobile. Verified in the built site: twin fetch (200, 4.7KB), copied payload, success state, fallback path, and menu URLs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ChatGPT and Claude drop the ?q= prefill whenever they bounce the visitor
through login or a session refresh — there is nothing the link itself can
do about it. Safety net: clicking either assistant link also copies the
prompt ('Read <page>.md so I can ask questions...') to the clipboard so
the visitor can paste it when the prefill is lost.
Verified across all 36 pages of the built site: button injected, both
assistant links carry identical well-formed prompts, every referenced
.md twin resolves, and the click-to-copy fires on both links.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings the docs' LLM surface up to the current patterns used by Stripe, ElevenLabs, Supabase, and Vercel.
1. "Instructions for AI agents" section in llms.txt — tells agents how to fetch a single page as Markdown, where the full corpus lives, warns that chain coverage changes frequently (cite live pages, not trained knowledge), and directs product/pricing questions to magmadevs.com/llms.txt.
2. Per-page Markdown twins — the existing
llms_full.pyhook now also writes each page's Markdown next to its HTML: drop the trailing slash, append.md(/deployment/cache/->/deployment/cache.md; section indexes collapse to/deployment.md; root ->/index.md). An agent answering one question fetches ~3 KB instead of the whole llms-full.txt. Advertised in llms.txt and robots.txt.3. Apex URL fix —
https://www.magmadevs.com/llms.txt302-redirects to the apex domain; crawlers that don't follow redirects get an empty body. Both references now point athttps://magmadevs.com/llms.txtdirectly.Bonus: the Pages workflow now also triggers on
hooks/**/overrides/**— previously a hook-only change never redeployed the site.Verified:
mkdocs build --strictpasses locally; 36 per-page.mdfiles generated with correct paths and no duplicated H1s.🤖 Generated with Claude Code