Public results and verification data for LemonMetrics.github.io — the community energy-per-token leaderboard for Lemonade local AI servers.
This repository is the results platform. It holds:
data/results/— community-submitted benchmark runs, each containingreport.json(normalized schema v1), rawpower.jsonlandbaseline.jsonlsamples, rawbench.json, andsummary.md. The raw power samples are the point: every number on the site is independently verifiable.site/— the static leaderboard (Astro → GitHub Pages).worker/— the Cloudflare Worker that powers community voting.scripts/validate_results.py— the CI validator every submission must pass.
The measurement harness itself lives in the separate lemonmetrics repository.
- Install and run the harness (docs). Each run writes a
data/results/<device>/<run>/directory withreport.json,power.jsonl, and friends. - Fork this repository and add your
data/results/...directory (you can do this entirely in the GitHub web UI — no clone needed). - Open a pull request. CI runs
scripts/validate_results.pyagainst the results directory; the schema must pass and power samples must look sane. - Once merged, the site rebuilds and your run appears on the leaderboard.
- Benchmark on AC power with
--runs 3or more. - Run the same model on one machine across backends for honest comparisons.
- Don't cherry-pick runs; submit what you measured. Methodology and energy math are documented in the harness docs.
# Validate the submitted results dataset
python3 scripts/validate_results.py data/results
# Build the static site (requires Node 22)
cd site && npm install && npm run buildThe site build reads data/results/ at build time and outputs static pages to site/dist/ ready for GitHub Pages.
ci.yml— validates submitted results on every PR.deploy-site.yml— builds the site and deploys to GitHub Pages onmain.
MIT — see LICENSE.
Running the public site has costs. Buy me a coffee if the leaderboard helps you — it goes toward hosting and the domain.