non-blocking i/o

.gitignore

@@ -36,6 +36,13 @@
 .idea
 build
 cmake-build-debug
+.cache
+
+# Benchmarking
+node_modules
+benchmark/results
+__pycache__
+**/package-lock.json
 
 # Secrets
 *.env

benchmark/README.md

@@ -0,0 +1,116 @@
+# Expresso benchmark suite
+
+Compares expresso against other HTTP frameworks and runtimes using servers
+that all mirror `example/main.cpp` — same routes, same payloads, same
+headers — so the measurements compare frameworks, not fixtures.
+
+## Layout
+
+```
+benchmark/
+├── servers/
+│   ├── shared/       # framework-agnostic fixtures (about payload, zip
+│   │                 # builder, mime map, parity headers)
+│   ├── node-http/    # dependency-free node:http mirror (runs on node/bun/deno)
+│   └── express/      # express.js mirror (pinned deps, npm ci)
+├── scripts/
+│   ├── frameworks.json     # registry of frameworks, scenarios, load settings
+│   ├── run_benchmarks.py   # orchestrator (python3 stdlib only)
+│   └── compare.py          # renders markdown reports from result JSON
+└── results/          # timestamped output, gitignored
+```
+
+## Prerequisites
+
+- [wrk](https://github.com/wg/wrk) (`brew install wrk` / `apt install wrk`)
+- python3 (stdlib only, no pip packages)
+- node ≥ 20 for the JavaScript servers; bun and deno are optional and are
+  skipped automatically when not installed
+- the expresso example built: `cmake -B build && cmake --build build`
+
+## Running
+
+```sh
+cd benchmark/scripts
+python3 run_benchmarks.py               # full run, all available frameworks
+python3 run_benchmarks.py --quick       # 2s smoke run
+python3 run_benchmarks.py --frameworks expresso,express --scenarios health
+python3 run_benchmarks.py --mode keepalive   # see caveats below
+```
+
+Each run writes `benchmark/results/<timestamp>_<mode>/` containing
+`results.json` (raw trials + environment metadata), `comparison.md`, and a
+log per server. Re-render or inspect old runs with
+`python3 compare.py ../results/<run>/results.json`.
+
+## Methodology and reproducibility
+
+- **Connection-close mode by default.** Expresso does not implement
+  keep-alive yet, so every response carries `connection: close`. All
+  frameworks are therefore driven with `Connection: close` so everyone pays
+  the same per-request TCP setup cost. `--mode keepalive` exists for
+  measuring the JavaScript servers against each other, but expresso numbers
+  in that mode are still close-mode numbers — don't compare across.
+- **TIME_WAIT draining.** Close-mode runs burn one ephemeral port per
+  request; on macOS especially, back-to-back runs exhaust ports and produce
+  garbage numbers (connect errors, 100x throughput collapse). The harness
+  waits for TIME_WAIT sockets to drop below a threshold between trials.
+- **The macOS close-mode wall (why trials are short on macOS).** Whichever
+  TCP peer closes first holds the TIME_WAIT. When the *client* (wrk) wins
+  the FIN race, that ephemeral port is pinned for 2×MSL and cannot be reused
+  toward the same server — and macOS only has ~16k ephemeral ports. A server
+  that closes even slightly late loses the FIN race often (measured: ~47% of
+  connections for node vs ~16% for expresso), so a long trial saturates the
+  client port space mid-run and throughput collapses: the same node server
+  measured 32.5k req/s in a 2s trial but 8.5k in an 8s trial from an
+  identical clean start. This is a client-side artifact, not server
+  performance. The harness therefore defaults to 2s × 5 trials on macOS
+  (`load.platform_overrides` in frameworks.json) so trials finish before the
+  wall; Linux keeps 8s × 3 (wider port range, `tcp_tw_reuse`). For
+  authoritative sustained close-mode numbers, run the suite on Linux.
+- **Warmup + median.** Every scenario gets a warmup run, then N timed trials
+  (default 3); the report shows the median. Trials with socket connect
+  errors, timeouts, or non-2xx responses are discarded and retried.
+- **Environment stamping.** Results embed the git commit (with dirty flag),
+  CPU, core count, OS, and tool versions. Compare runs only when those match.
+- **Pinned dependencies.** The express server pins exact versions and
+  installs with `npm ci` against the committed lockfile.
+- Benchmark on an idle machine, on AC power, and prefer comparing numbers
+  from the same run over numbers from different days.
+
+## Scenarios
+
+| Name | Route | What it measures |
+|---|---|---|
+| health | `GET /health` | per-request framework overhead |
+| about-json | `GET /about` | ~1 KB JSON serialization |
+| static-png | `GET /coffee.png` | file serving (expresso uses sendfile) |
+| post-print | `POST /print` | body parsing + JSON echo |
+
+`/download` (zip) and `/pictures` (directory listing) are implemented by
+every server for API parity but not timed: the zip route's output size
+differs per archiver, which would make transfer-bound comparisons
+meaningless.
+
+## Adding a framework
+
+1. Create `benchmark/servers/<name>/` with a server that mirrors
+   `example/main.cpp`. Reuse `servers/shared/` for the about payload, mime
+   types, parity headers, and the zip builder — responses must stay
+   byte-comparable. The server must read `PORT` (and optionally `ASSETS`)
+   from the environment.
+2. Add an entry to `frameworks.json`: `name`, `cwd` (repo-root relative),
+   `start` command, optional `probe` (binary that must exist, else the
+   framework is skipped), optional `install`/`installed_marker` and `build`.
+
+Adding another runtime for an existing server (like bun/deno for node-http)
+is just another registry entry with a different `start` command.
+
+## Known caveats
+
+- Header parity is approximate: each framework adds its own incidental
+  headers (`X-Powered-By`, `Date`, etags), so tiny-response scenarios have a
+  few dozen bytes of variance.
+- The JavaScript mirrors log `POST /print` bodies like the example does;
+  all server stdout goes to the per-run log files so console throughput
+  doesn't skew anyone.

benchmark/scripts/compare.py

@@ -0,0 +1,134 @@
+#!/usr/bin/env python3
+"""Render a markdown comparison from benchmark result JSON files.
+
+Used automatically by run_benchmarks.py at the end of a run, and usable
+standalone to re-render or diff older runs:
+
+    python3 compare.py ../results/20260719T120000Z_close/results.json
+"""
+
+import argparse
+import json
+from pathlib import Path
+
+
+def _scenario_names(results):
+    names = []
+    for framework in results["frameworks"].values():
+        for name in framework.get("scenarios", {}):
+            if name not in names:
+                names.append(name)
+    return names
+
+
+def _framework_names(results):
+    return list(results["frameworks"].keys())
+
+
+def _median_rate(results, framework, scenario):
+    data = (
+        results["frameworks"]
+        .get(framework, {})
+        .get("scenarios", {})
+        .get(scenario, {})
+    )
+    rate = data.get("requests_per_second")
+    return rate["median"] if rate else None
+
+
+def render_markdown(results, baseline=None):
+    metadata = results["metadata"]
+    frameworks = _framework_names(results)
+    scenarios = _scenario_names(results)
+    if baseline not in frameworks:
+        baseline = frameworks[0] if frameworks else None
+
+    lines = ["# Benchmark comparison", ""]
+    lines.append(f"- **Date:** {metadata['timestamp_utc']}")
+    lines.append(f"- **Commit:** {metadata['git_commit']}")
+    lines.append(f"- **Machine:** {metadata['cpu']} ({metadata['logical_cores']} cores), {metadata['platform']}")
+    load = metadata["load"]
+    lines.append(
+        f"- **Load:** wrk -t{load['threads']} -c{load['connections']} "
+        f"-d{load['duration_seconds']}s, {load['trials']} trial(s), "
+        f"connection mode: {load['connection_mode']}"
+    )
+    versions = ", ".join(
+        f"{tool} {version}"
+        for tool, version in metadata["tool_versions"].items()
+        if version
+    )
+    lines.append(f"- **Tools:** {versions}")
+    lines.append("")
+
+    lines.append("## Throughput (median requests/sec, higher is better)")
+    lines.append("")
+    header = ["Scenario"] + [
+        f"{name}{' (baseline)' if name == baseline else ''}" for name in frameworks
+    ]
+    lines.append("| " + " | ".join(header) + " |")
+    lines.append("|" + "---|" * len(header))
+    for scenario in scenarios:
+        base_rate = _median_rate(results, baseline, scenario)
+        row = [scenario]
+        for framework in frameworks:
+            rate = _median_rate(results, framework, scenario)
+            if rate is None:
+                skipped = results["frameworks"][framework].get("skipped")
+                row.append("skipped" if skipped else "-")
+            elif base_rate and framework != baseline:
+                row.append(f"{rate:,.0f} ({rate / base_rate:.2f}x)")
+            else:
+                row.append(f"{rate:,.0f}")
+        lines.append("| " + " | ".join(row) + " |")
+    lines.append("")
+
+    lines.append("## Latency (median of trials, milliseconds)")
+    lines.append("")
+    lines.append("| Scenario | " + " | ".join(frameworks) + " |")
+    lines.append("|" + "---|" * (len(frameworks) + 1))
+    for scenario in scenarios:
+        row = [scenario]
+        for framework in frameworks:
+            data = (
+                results["frameworks"][framework]
+                .get("scenarios", {})
+                .get(scenario, {})
+            )
+            if "latency_avg_ms" in data:
+                row.append(
+                    f"avg {data['latency_avg_ms']}, p99 {data['latency_p99_ms']}"
+                )
+            else:
+                row.append("-")
+        lines.append("| " + " | ".join(row) + " |")
+    lines.append("")
+
+    skipped = {
+        name: framework["skipped"]
+        for name, framework in results["frameworks"].items()
+        if "skipped" in framework
+    }
+    if skipped:
+        lines.append("## Skipped")
+        lines.append("")
+        for name, reason in skipped.items():
+            lines.append(f"- **{name}**: {reason}")
+        lines.append("")
+
+    return "\n".join(lines)
+
+
+def main():
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument("results", nargs="+", help="results.json file(s) to render")
+    parser.add_argument("--baseline", default="node-http")
+    args = parser.parse_args()
+
+    for results_path in args.results:
+        results = json.loads(Path(results_path).read_text())
+        print(render_markdown(results, baseline=args.baseline))
+
+
+if __name__ == "__main__":
+    main()

benchmark/scripts/frameworks.json

@@ -0,0 +1,88 @@
+{
+  "port": 3210,
+  "load": {
+    "threads": 4,
+    "connections": 64,
+    "duration_seconds": 8,
+    "warmup_seconds": 2,
+    "trials": 3,
+    "connection_mode": "close",
+    "time_wait_threshold": 2000,
+    "time_wait_max_wait_seconds": 60,
+    "platform_overrides": {
+      "darwin": {
+        "duration_seconds": 2,
+        "warmup_seconds": 1,
+        "trials": 5
+      }
+    }
+  },
+  "frameworks": [
+    {
+      "name": "expresso",
+      "language": "C++",
+      "probe": null,
+      "cwd": "build",
+      "build": ["cmake", "--build", ".", "-j"],
+      "start": ["./server"]
+    },
+    {
+      "name": "node-http",
+      "language": "JavaScript (node)",
+      "probe": "node",
+      "cwd": "benchmark/servers/node-http",
+      "start": ["node", "server.js"]
+    },
+    {
+      "name": "express",
+      "language": "JavaScript (node)",
+      "probe": "node",
+      "cwd": "benchmark/servers/express",
+      "install": ["npm", "ci"],
+      "installed_marker": "node_modules",
+      "start": ["node", "server.js"]
+    },
+    {
+      "name": "node-http-bun",
+      "language": "JavaScript (bun)",
+      "probe": "bun",
+      "cwd": "benchmark/servers/node-http",
+      "start": ["bun", "server.js"]
+    },
+    {
+      "name": "node-http-deno",
+      "language": "JavaScript (deno)",
+      "probe": "deno",
+      "cwd": "benchmark/servers/node-http",
+      "start": ["deno", "run", "--allow-net", "--allow-read", "--allow-env", "server.js"]
+    }
+  ],
+  "scenarios": [
+    {
+      "name": "health",
+      "description": "Tiny plain-text response; measures per-request framework overhead",
+      "method": "GET",
+      "path": "/health"
+    },
+    {
+      "name": "about-json",
+      "description": "~1 KB JSON serialization",
+      "method": "GET",
+      "path": "/about"
+    },
+    {
+      "name": "static-png",
+      "description": "1.4 MB static file; measures the file-serving path",
+      "method": "GET",
+      "path": "/coffee.png"
+    },
+    {
+      "name": "post-print",
+      "description": "JSON body parse + echo",
+      "method": "POST",
+      "path": "/print",
+      "body": "{\"benchmark\": true, \"suite\": \"expresso\"}",
+      "content_type": "application/json"
+    }
+  ]
+}