Replace provider-native web search with a dedicated in-enclave /v1/web_search endpoint

CLAUDE.md

@@ -12,6 +12,7 @@ The repo must provide a stable AWS Nitro PCR when the code doesn't change in ord
 │   ├── llm_backend.py       # LLM provider routing via LangChain, HTTP client management
 │   ├── image_generation.py  # Endpoint-based image gen (/images/generations): request shaping, URL→inline-bytes, signed responses
 │   ├── tee_manager.py       # TEE key generation, nitriding registration, response signing
+│   ├── web_search.py        # In-enclave web search: Exa client, execution, result formatting
 │   ├── model_registry.py    # Model config and per-token pricing
 │   ├── definitions.py       # On-chain addresses, network IDs, payment amounts
 │   ├── facilitator_api.py   # x402 facilitator API client
@@ -69,6 +70,9 @@ API keys (injected at runtime via POST /v1/keys — do NOT bake into the image):
 - `ARK_API_KEY` (BytePlus / ByteDance ModelArk; injected as `bytedance_api_key`)
 - `NOUS_API_KEY` (Nous Research / Nous Portal; injected as `nous_api_key`)
 - `ZAI_API_KEY` (Z.ai Model API; injected as `zai_api_key`)
+- `EXA_API_KEY` (Exa search; injected as `exa_api_key`) — backs the in-enclave
+  `/v1/web_search` endpoint, not an LLM provider. Without it the endpoint
+  returns 503 and `/health` reports `web_search_enabled: false`.
 
 Server configuration:
 - `API_SERVER_PORT` (default: 8000)
@@ -92,18 +96,20 @@ Server configuration:
 - **`llm_backend.py`**: LangChain model instantiation, HTTP client management, provider routing from model name
 - **`model_registry.py`**: Maps model names to providers and per-token USD pricing (used by dynamic cost calculator)
 - **`definitions.py`**: On-chain constants (addresses, network IDs, payment amounts) — configure here for your deployment
+- **`web_search.py`**: Exa HTTP client, search execution, and result formatting/citation extraction (serves `/v1/web_search`)
 - **`util.py`**: `dynamic_session_cost_calculator` converts actual token usage to x402 payment amounts
 
 ### API Endpoints
 
 | Endpoint | Purpose |
 |----------|---------|
-| `/health` | Health check (status, version, tee_enabled) |
+| `/health` | Health check (status, version, tee_enabled, web_search_enabled) |
 | `/signing-key` | TEE public key (PEM) and tee_id |
 | `/enclave/attestation` | Nitro attestation document (served by nitriding) |
 | `/v1/keys` | One-time API key injection (POST, loopback-only) |
 | `/v1/completions` | Text completion (signed) |
 | `/v1/chat/completions` | Chat completion with tool support (signed) |
+| `/v1/web_search` | In-enclave Exa web search (signed, flat per-search price) |
 
 ### TEE Integration
 
@@ -144,6 +150,38 @@ skipped rather than dereferenced in the enclave). Per-provider request quirks
 `model_registry.py`. These models are billed a flat per-image
 price (see `per_image_price_usd`), not per token.
 
+### Web Search
+
+Web search is a dedicated endpoint — `POST /v1/web_search` — not a chat feature.
+It does NOT use any provider's native web search (OpenAI/Anthropic/Google/xAI
+all have one; those were removed), and the gateway runs no tool loop of its own:
+the client advertises a `web_search` function tool to its model, calls this
+endpoint when the model invokes it, and feeds the returned `content` back as the
+tool result. The search runs inside the enclave against Exa (`web_search.py`),
+so a query rides the same encrypted OHTTP channel as chat and is never visible
+to the relay or the gateway operator. Points to keep in mind:
+
+- **The chat/completions `web_search` request flag is a deprecated no-op.** It
+  is still accepted (and still part of the signed request hash when sent) so
+  old clients' requests parse and verify, but it binds nothing and bills
+  nothing.
+- **Every text model can search** — the tool lives in the client, so this is
+  purely a question of function calling, not of provider search support.
+- **Request/response**: `{"query", "num_results"?, "recency_days"?}` in;
+  `content` (model-ready numbered results), `citations` (structured sources),
+  and the standard `tee_*` signing fields out. The request hash covers the
+  canonical (sorted-keys) JSON body; the output hash covers `content`.
+- **Reachable through OHTTP**: the inner payload's `endpoint` field
+  (`"web_search"`) routes the sealed request; absent means chat, so existing
+  OHTTP clients are unaffected. Billing flows through the same outer
+  cost-header / billing-frame channel the relay already consumes.
+- **Billing is one flat rate** (`WEB_SEARCH_PRICE_USD`) per search that reached
+  Exa, settled from the response's `opengradient` block like every paid
+  endpoint. Validation failures (400/503) and Exa failures (502) return no cost
+  block and are never settled. A search that ran but matched nothing IS billed.
+- Exa's self-reported `costDollars` is logged for margin reconciliation only;
+  settlement never depends on it.
+
 ## Verification Examples
 
 - `examples/verify_attestation.py` — Validates AWS Nitro attestation documents against the root CA

README.md

@@ -24,9 +24,10 @@ The gateway solves this by running inside a hardware-isolated Nitro Enclave wher
 - **Request integrity** - SHA256 hash of original request included in signed response
 - **Streaming support** - SSE streaming for chat completions
 - **Tool/function calling** - Full support for LLM tool use
-- **Native web search** - Opt-in `web_search` flag enables each provider's built-in
-  web search (OpenAI, Anthropic, Google, xAI); searches are billed per search on top
-  of token usage
+- **In-enclave web search** - Dedicated `/v1/web_search` endpoint (backed by
+  Exa) for clients running their own tool loop. The query never leaves the
+  enclave except to the search backend; every search is billed at one flat
+  per-search rate
 
 ## Supported Models
 
@@ -90,21 +91,29 @@ curl -X POST http://127.0.0.1:8000/v1/completions \
     "prompt": "Explain quantum computing in one sentence"
   }'
 
-# Native web search (set "web_search": true on any supported model)
-curl -X POST http://127.0.0.1:8000/v1/chat/completions \
+# Web search (dedicated endpoint; advertise a `web_search` tool to your model
+# and call this when the model invokes it)
+curl -X POST http://127.0.0.1:8000/v1/web_search \
   -H "Content-Type: application/json" \
   -d '{
-    "model": "claude-sonnet-4-5",
-    "messages": [{"role": "user", "content": "What happened in the news today?"}],
-    "web_search": true
+    "query": "What happened in the news today?",
+    "num_results": 6
   }'
 ```
 
-> **Web search & billing.** When `web_search` is `true`, the gateway enables the
-> provider's built-in web search (OpenAI, Anthropic, Google, xAI). Each search is
-> billed per search on top of token usage — at the provider's list price — and the
-> charge is reflected in the dynamically-settled x402 amount. Providers without
-> native web search (e.g. ByteDance/ModelArk) ignore the flag and are not charged.
+> **Web search & billing.** `/v1/web_search` runs the search inside the enclave
+> against Exa — it does not use any provider's built-in search, and the gateway
+> runs no tool loop: feed the returned `content` back to your model as the tool
+> result, and show `citations` to your user. The query never leaves the TEE
+> except to the search backend, and via OHTTP (inner `"endpoint":
+> "web_search"`) it is also invisible to the relay.
+>
+> Each search is one flat price, settled via x402 from the response's
+> `opengradient` block like any other paid endpoint; failed searches return no
+> cost block and are not charged. The response is signed with the same `tee_*`
+> fields as chat (request hash over the canonical JSON body, output hash over
+> `content`). Requires `EXA_API_KEY` to be injected; check `web_search_enabled`
+> on `/health`. The old chat-request `web_search` flag is a deprecated no-op.
 
 ## Deployment to Nitro Enclave
 

scripts/run-enclave.sh

@@ -92,6 +92,8 @@ if [ -f "$ENV_FILE" ]; then
         ARK_API_KEY="$(grep -E '^ARK_API_KEY=' "$ENV_FILE" | cut -d'=' -f2-)"
         NOUS_API_KEY="$(grep -E '^NOUS_API_KEY=' "$ENV_FILE" | cut -d'=' -f2-)"
         ZAI_API_KEY="$(grep -E '^ZAI_API_KEY=' "$ENV_FILE" | cut -d'=' -f2-)"
+        # Backs the in-enclave `web_search` tool (Exa), not an LLM provider.
+        EXA_API_KEY="$(grep -E '^EXA_API_KEY=' "$ENV_FILE" | cut -d'=' -f2-)"
 
         # FACILITATOR_URL is used for both x402 payment verification and the heartbeat relay.
         # HEARTBEAT_CONTRACT_ADDRESS and TEE_HEARTBEAT_INTERVAL are optional heartbeat parameters.
@@ -110,6 +112,7 @@ if [ -f "$ENV_FILE" ]; then
             --arg bytedance "$ARK_API_KEY" \
             --arg nous "$NOUS_API_KEY" \
             --arg zai "$ZAI_API_KEY" \
+            --arg exa "$EXA_API_KEY" \
             --arg hb_contract "$HEARTBEAT_CONTRACT_ADDRESS" \
             --arg facilitator "$FACILITATOR_URL" \
             --arg hb_interval "$TEE_HEARTBEAT_INTERVAL" \
@@ -120,7 +123,8 @@ if [ -f "$ENV_FILE" ]; then
                 xai_api_key: $xai,
                 bytedance_api_key: $bytedance,
                 nous_api_key: $nous,
-                zai_api_key: $zai
+                zai_api_key: $zai,
+                exa_api_key: $exa
             }
             + if $hb_contract != "" then {heartbeat_contract_address: $hb_contract} else {} end
             + if $facilitator != "" then {facilitator_url: $facilitator} else {} end
@@ -151,7 +155,7 @@ if [ -f "$ENV_FILE" ]; then
 
         # Clear key variables from this shell immediately after use
         unset OPENAI_API_KEY GOOGLE_API_KEY ANTHROPIC_API_KEY XAI_API_KEY ARK_API_KEY
-        unset NOUS_API_KEY ZAI_API_KEY
+        unset NOUS_API_KEY ZAI_API_KEY EXA_API_KEY
         unset HEARTBEAT_CONTRACT_ADDRESS FACILITATOR_URL TEE_HEARTBEAT_INTERVAL
     fi
 else

tee_gateway/__main__.py

@@ -22,11 +22,13 @@
     DEFAULT_HEARTBEAT_INTERVAL,
 )
 from tee_gateway.llm_backend import get_provider_config, set_provider_config
+from tee_gateway.web_search import web_search_available
 from tee_gateway.heartbeat import create_heartbeat_service
 from tee_gateway.controllers.ohttp_controller import (
     create_anonymous_chat_completion,
     get_hpke_config,
 )
+from tee_gateway.controllers.web_search_controller import create_web_search
 
 from x402.http import FacilitatorConfig, HTTPFacilitatorClientSync, PaymentOption
 from x402.http.middleware.flask import payment_middleware
@@ -51,6 +53,7 @@
     COMPLETIONS_OPG_SESSION_MAX_SPEND,
     FACILITATOR_URL,
     OHTTP_OPG_SESSION_MAX_SPEND,
+    WEB_SEARCH_OPG_SESSION_MAX_SPEND,
 )
 
 # ---------------------------------------------------------------------------
@@ -321,6 +324,29 @@ def _init_payment_middleware(facilitator_url: str) -> None:
             mime_type="application/json",
             description="Completion",
         ),
+        "POST /v1/web_search": RouteConfig(
+            accepts=[
+                PaymentOption(
+                    scheme="upto",
+                    pay_to=EVM_PAYMENT_ADDRESS,
+                    price=AssetAmount(
+                        amount=WEB_SEARCH_OPG_SESSION_MAX_SPEND,
+                        asset=BASE_MAINNET_OPG_ADDRESS,
+                        extra={
+                            "name": "OpenGradient",
+                            "version": "1",
+                            "assetTransferMethod": "permit2",
+                        },
+                    ),
+                    network=BASE_MAINNET_NETWORK,
+                ),
+            ],
+            extensions={
+                **declare_erc20_approval_gas_sponsoring_extension(),
+            },
+            mime_type="application/json",
+            description="Web search",
+        ),
         "POST /v1/ohttp": RouteConfig(
             accepts=[
                 PaymentOption(
@@ -395,6 +421,7 @@ def set_provider_keys():
             bytedance_api_key=body.get("bytedance_api_key") or None,
             nous_api_key=body.get("nous_api_key") or None,
             zai_api_key=body.get("zai_api_key") or None,
+            exa_api_key=body.get("exa_api_key") or None,
         )
         set_provider_config(provider_config)
 
@@ -460,6 +487,9 @@ def _set(val: str | None) -> str:
         logger.info(
             "  zai_api_key                 : %s", _set(provider_config.zai_api_key)
         )
+        logger.info(
+            "  exa_api_key (web search)    : %s", _set(provider_config.exa_api_key)
+        )
         logger.info("  facilitator_url             : %s", facilitator_url)
         logger.info(
             "  heartbeat_contract_address  : %s",
@@ -503,6 +533,7 @@ def _set(val: str | None) -> str:
             "status": "ok",
             "providers_initialized": providers_set,
             "heartbeat_enabled": heartbeat_config is not None,
+            "web_search_enabled": bool(provider_config.exa_api_key),
         }
     ), 200
 
@@ -517,6 +548,10 @@ def health():
         "tee_enabled": True,
         "uptime_seconds": int(time.time() - _started_at),
         "providers": providers,
+        # Whether /v1/web_search can serve searches (an Exa key was injected).
+        # Not a provider capability — the search endpoint has no model — so it
+        # is reported separately from `providers`.
+        "web_search_enabled": web_search_available(),
         "facilitator_url": _active_facilitator_url,
         "price_feed": _price_feed.get_status(),
     }, 200
@@ -567,6 +602,13 @@ def create_app():
         "/v1/ohttp/config", "ohttp-config", get_hpke_config, methods=["GET"]
     )
 
+    # Dedicated in-enclave web search (Exa). Mounted outside the OpenAI spec
+    # like the other gateway-specific endpoints; the client's tool loop calls
+    # it when its model asks to search (see web_search_controller.py).
+    app.app.add_url_rule(
+        "/v1/web_search", "web-search", create_web_search, methods=["POST"]
+    )
+
     # Initialize TEE here so it runs under both Gunicorn and direct execution.
     # This is the single TEEKeyManager instance — the same key both registers
     # with nitriding and signs all LLM responses.
@@ -600,15 +642,21 @@ def create_app():
 
 @application.before_request
 def _check_pricing_ready():
-    if request.path not in ("/v1/chat/completions", "/v1/completions", "/v1/ohttp"):
+    if request.path not in (
+        "/v1/chat/completions",
+        "/v1/completions",
+        "/v1/ohttp",
+        "/v1/web_search",
+    ):
         return
     try:
         _price_feed.get_price()
     except ValueError as exc:
         logger.warning("Rejecting inference request — price feed unavailable: %s", exc)
         return jsonify({"error": f"Pricing unavailable: {exc}"}), 503
 
-    if request.path == "/v1/ohttp":
+    # OHTTP carries its model inside the ciphertext; web search has no model.
+    if request.path in ("/v1/ohttp", "/v1/web_search"):
         return
 
     body = request.get_json(silent=True, cache=True) or {}

tee_gateway/config.py

@@ -21,7 +21,7 @@
 
 @dataclass(frozen=True)
 class ProviderConfig:
-    """API keys for each supported LLM provider."""
+    """API keys for each supported LLM provider, plus the web-search backend."""
 
     openai_api_key: Optional[str] = None
     anthropic_api_key: Optional[str] = None
@@ -30,6 +30,11 @@ class ProviderConfig:
     bytedance_api_key: Optional[str] = None
     nous_api_key: Optional[str] = None
     zai_api_key: Optional[str] = None
+    # Exa, which backs the `web_search` tool the gateway executes inside the
+    # enclave for every model (see web_search.py). Not an LLM provider, so it is
+    # deliberately absent from initialized_providers() — /health reports it
+    # separately as `web_search_enabled`.
+    exa_api_key: Optional[str] = None
 
     def initialized_providers(self) -> list[str]:
         """Return provider names whose API key is set (non-empty)."""