Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/deploy-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ jobs:
run: |
cd webui && npm ci && npm run build
cd ..
npx playwright test tests/e2e/pages-chat.spec.ts tests/e2e/failover-dual-endpoint.spec.ts tests/e2e/model-picker.spec.ts tests/e2e/message-actions.spec.ts tests/e2e/health-panel.spec.ts tests/e2e/turnstile-gate.spec.ts tests/e2e/catalog-explorer.spec.ts tests/e2e/export-session.spec.ts tests/e2e/hash-routing.spec.ts tests/e2e/import-session.spec.ts tests/e2e/shortcuts-sheet.spec.ts tests/e2e/streaming-polish.spec.ts
npx playwright test tests/e2e/pages-chat.spec.ts tests/e2e/failover-dual-endpoint.spec.ts tests/e2e/model-picker.spec.ts tests/e2e/message-actions.spec.ts tests/e2e/health-panel.spec.ts tests/e2e/turnstile-gate.spec.ts tests/e2e/catalog-explorer.spec.ts tests/e2e/export-session.spec.ts tests/e2e/hash-routing.spec.ts tests/e2e/import-session.spec.ts tests/e2e/shortcuts-sheet.spec.ts tests/e2e/streaming-polish.spec.ts tests/e2e/compare-mode.spec.ts tests/e2e/vision-attach.spec.ts tests/e2e/tier-settings.spec.ts

- name: Run live Pages chat e2e (real proxy)
env:
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,7 @@ When reviewing or submitting changes:
14. **Bootstrap merge:** `FailoverProvider` must use `loadRuntimeConfig()` (merged `chat_proxy.json` endpoints), not raw `readRuntimeConfig()` alone — stale single-endpoint localStorage hides secondaries until cleared (see `docs/CAVEATS.md`)
15. **Cloudflare auth 10000:** regenerate CF API token with Workers Scripts Edit; when deploy skips, `WORKER_URL` secret keeps Pages building (see `docs/solutions/workflow-issues/github-pages-webui-deploy-and-secrets.md`)
16. **Render deploy-mode YAML:** omit `DATABASE_URL` env var entirely — do not set empty string; deploy generator excludes `database_url` and `allowed_routes`
17. murm-ui streaming flicker: keep `webui/patches/murm-ui+0.2.0.patch` (plain-text tail until complete) via `postinstall` patch-package — bump alone was insufficient (see `docs/solutions/tooling-decisions/murm-ui-streaming-plaintext-tail-patch.md`)

## Compound Engineering

Expand Down
4 changes: 4 additions & 0 deletions CONCEPTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ Shared vocabulary for the static chat gateway and Python library.
| **Session export** | Sidebar menu download of current chat as Markdown or JSON (client-only; no server copy) |
| **Hash session link** | URL `#/chat/{sessionId}` restores a session from local IndexedDB on the same browser/profile |
| **Conversation import** | File-picker restore of exported Markdown/JSON into a new local session (symmetry with session export) |
| **Provider tier** | User-ordered stage in the omnifail stack (quality API, headless web UI, SearXNG discovery, proxy failover) |
| **Compare mode** | Two-column chat UI: same prompt to two independently configured sources |
| **Web UI provider tier** | Optional headless-browser route against free web chat UIs; requires user-run runner; off by default on public demo |
| **SearXNG discovery tier** | Metasearch pass to suggest candidate free web chat URLs when API/web tiers exhaust |

## Learnings index

Expand Down
9 changes: 5 additions & 4 deletions STRATEGY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: llm-fallbacks
last_updated: 2026-07-24
last_updated: 2026-07-25
---

# Strategy
Expand All @@ -15,7 +15,7 @@ Developers want free LLM chat without manually tracking which models exist, whic

Endpoint bootstrap is three-layer: Pages CI writes `docs/config.js`, the browser fetches `chat_proxy.json` at runtime, and the committed artifact lists dual proxy URLs for failover.

High availability on free tiers means **best-effort failover with cold-start penalties**, not paid uptime. We say that plainly.
High availability on free tiers means **best-effort failover with cold-start penalties**, not paid uptime. Exhaust free, legal routes the user opted into before giving up; cold starts, quotas, and ToS limits still apply. We say that plainly.

## Who it's for

Expand All @@ -33,13 +33,13 @@ Open-source builders and power users who want a demo-quality free LLM gateway ti

### Static public chat (GitHub Pages)

Minimal chat SPA as the repo homepage. Uses `free_models.json` for the model browser. Calls proxies only.
Static chat SPA as the repo homepage. Uses `free_models.json` for the model browser. Zero-config path uses ranked catalog + edge/container proxies only. Optional BYOK and user-run companions never embed repo-owned keys. Demo job: make ranked free failover tangible — not become Open WebUI.

_Keeps secrets off the static surface and makes the project tangible to visitors._

### Edge + container proxy HA

Cloudflare Worker primary (CORS, guest auth, rate limits, short fallback chain) plus **Render LiteLLM** as v1 secondary. Both driven by generated configs. Secondary redeploy via Render API when deploy hooks are unavailable; `chat_proxy.json` preserves dual endpoints across Worker-only CI runs.
Cloudflare Worker primary (CORS, guest auth, rate limits, short fallback chain) plus **Render LiteLLM** as v1 secondary. Both driven by generated configs. Secondary redeploy via Render API when deploy hooks are unavailable; `chat_proxy.json` preserves dual endpoints across Worker-only CI runs. Public $0 HA remains Worker + Render LiteLLM; user-run runners are opt-in power-user extensions, not part of the dual-proxy HA story.

_Runtime routing and keys cannot live in the browser; reuses `litellm_config_free.yaml` and `free` alias work._

Expand All @@ -61,6 +61,7 @@ _Without a living remediation track, operator runbooks drift from production._
- Full TypeScript port of llm-fallbacks discovery
- True multi-region DNS HA on $0
- Browser-direct provider calls with repo-owned keys on the public homepage
- Agent gateway features (MCP marketplace, tool execution loops, cloud session sync) — demo stays display/routing-first

## Messaging

Expand Down
4 changes: 4 additions & 0 deletions docs/CAVEATS.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ Honest limits for the public chat demo and the Python library.
| **Routing headers** | The routing chip reads `x-llm-fallbacks-endpoint` and LiteLLM headers from proxy responses. After edge changes, redeploy the Worker (`Deploy Proxies` workflow) for production to expose them. |
| **Turnstile** | Optional bot check when `TURNSTILE_SECRET` is set on the Worker and `turnstileSiteKey` is in `docs/config.js`. Skipped in local dev when secrets are absent. |
| **Hash session links** | `#/chat/{id}` only works when that session exists in your browser’s IndexedDB. Copy the link on another device or after clearing site data and you get a new empty chat — use **Export as Markdown/JSON** to share transcripts. |
| **Provider tiers vs free-tier limits** | The **Tiers** panel is the omnifail *route stack* (direct/BYOK → optional local runner → SearXNG → cloud proxy). That is separate from cloud *free-tier* rate limits and quotas on OpenRouter / Workers. Reordering tiers changes which route we try first; it does not raise provider quotas. |
| **Bootstrap merge** | Chat endpoints come from `loadRuntimeConfig()` (page `config.js` + `chat_proxy.json` merged with localStorage). If you once saved a single endpoint, that stale localStorage value can hide newer secondaries until you clear site data or re-save Server settings — see AGENTS.md pitfall 14. |
| **Web-UI runner & SearXNG** | Both are **opt-in** and empty by default on the public homepage. You run them locally; you own target-site and SearXNG terms of service. We do not harvest credentials. Exhausting enabled tiers still fails honestly — this is best-effort free HA, not “never fail.” |
| **Vision export** | Session Markdown/JSON export is **text-only**. Attached images are not serialized into export files (thumbnails stay in IndexedDB until you clear site data). |

## Library and CI

Expand Down
108 changes: 108 additions & 0 deletions docs/assets/chat.css

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading