bddfirst

Specs describe. Tests prove. bddfirst compiles Gherkin behavior specs into executable tests — what you specified is what gets verified.

English | 简体中文

What this is

bddfirst is a CLI tool that makes behavior specifications executable. You write Gherkin .feature files — Given/When/Then scenarios readable by both humans and AI agents — and bddfirst compiles them into executable tests (currently targeting pytest).

The .feature files are the first-class citizen: they are the single source of truth for your system's behavior, and everything else is derived from them. The step bodies are filled in by you or your AI coding agent, and the generated tests are ordinary pytest files — nothing stands between your specs and your test framework.

On top of that, bddfirst adds two guards that ordinary test suites don't have:

• Drift check — bddfirst check detects, at scenario granularity, when specs and tests move apart.
• Expected-value audit — bddfirst audit verifies that the expected values written in Then steps are actually asserted, catching tests that prove nothing.

Why it's needed

Descriptive specs can't be verified. Spec-Driven Development (SDD) produces markdown documents. Markdown is prose: it can't run, it can't fail, and nobody gets notified when it stops matching reality. In the AI-coding era this gets worse, not better — AI writes code much faster than humans update docs, so spec-code drift becomes the default state. Even coverage can quietly lie: AI can push the number up while quality drops, because the human eyes that used to verify things are gone. A document nobody can execute binds nothing.

The spec-driven community has noticed this gap. In the OpenSpec community a natural question was raised: OpenSpec specs already contain BDD-style scenarios — so why not bridge the two worlds and make those scenarios executable? Then writing the spec first would also mean writing the tests first, adding exactly the layer of validation SDD is missing (the discussion behind this project: OpenSpec #508).

That is exactly what bddfirst does.

Then why not Cucumber or pytest-bdd? Gherkin itself is not the problem — Given/When/Then is one of the best spec formats for both humans and LLMs. The problem is the glue wrapped around it: hand-written step definitions rot just like documentation, and a separate runtime inserts itself between your tests and your test framework. In the AI era the "natural language → code" glue is work an AI agent does for free. bddfirst compiles Gherkin directly to native tests and drops both glue layers.

What you get

• Behavior specs as the single source of truth — .feature files govern; tests derive.
• Native test output — compiled tests run directly on your project's test framework (pytest today), with no BDD runtime in between.
• A drift gate — scenario anchors + a two-way check pinpoint exactly which scenario changed, disappeared, or got detached from its test.
• A trap for self-proving tests — the expected-value audit catches tests reverse-engineered from the implementation ("AI reads the code, writes an assertion that matches, calls it done").
• An OpenSpec bridge — generate behavior skeletons from OpenSpec scenarios and reconcile every SDD requirement against executable behavior.
• Agent-ready — ships a Claude Code skill + slash commands, and an AGENTS.md for every other coding agent.

Quick start

Prerequisites: Python ≥3.10 and uv. bddfirst generates pytest files but does not ship pytest — your project installs it itself.

uv tool install git+https://github.com/cuikexi/bddfirst   # PyPI release pending
cd your-project
uv add --dev pytest
bddfirst init                # behaviors/ + Claude Code skill + AGENTS.md
# developing from source? use: uv tool install --editable /path/to/bddfirst

Want your AI to do it? Paste this prompt into your coding agent

Install bddfirst in this project and set it up for me.
Follow these steps in order, and stop where a step tells you to stop.

1. RUNTIME. Run `python3 --version` (or `python --version`). bddfirst needs
   Python 3.10+. If it is missing, stop and tell me.
2. UV. Run `uv --version`. If uv is missing, stop and point me to
   https://docs.astral.sh/uv/ — do not install uv without my confirmation.
3. INSTALL. Run `uv tool install git+https://github.com/cuikexi/bddfirst`,
   then verify with `bddfirst --version`.
4. PROJECT. In my project root: if pytest is not already a dev dependency, add
   it with this project's package manager (for uv: `uv add --dev pytest`).
   Then run `bddfirst init`.
5. VERIFY. Run `bddfirst list` (an empty behavior map is fine at this stage).
   Tell me what init actually printed and which files were created — do not
   assume.

Write a behavior spec:

# behaviors/checkout.feature
Feature: checkout discount

  Scenario: threshold coupon deduction
    Given the order amount is 100
    When the SAVE10 coupon is applied
    Then the payable amount is 90

Compile, implement, verify:

bddfirst build         # -> tests/behaviors/checkout/test_checkout.py (skeleton)
bddfirst status        # pending backlog with feature line numbers

# implement the skeleton (human or AI agent): fill the test body, remove the @pytest.mark.skip
uv run pytest -v       # green

bddfirst check         # exit 0: feature <-> test pairing intact
bddfirst audit         # exit 0: the expected value "90" is really asserted

A complete runnable example lives in examples/shop (spec layer + behavior layer + implemented scenarios).

What it looks like

bddfirst list on the example project (Gherkin supports any spoken language — this feature is written in Chinese):

cart  behaviors/cart/cart.feature
  ✓ 添加商品  implemented  @core @spec(cart#add-item)
  ○ 批量添加  skeleton  @spec(cart#bulk-add)

total: 1 capability(ies) · 1 feature(s) · 3 scenario(s) (1 implemented / 2 skeleton)

bddfirst check and bddfirst audit as commit gates:

no drift: features and tests are in sync
audit done: 1 implemented scenario(s), 2 skeleton(s) skipped, 0 unauditable, 0 missing expected value(s)

bddfirst coverage reconciling OpenSpec specs with the behavior layer:

SDD specs (openspec/specs/): 2 requirement(s)
behavior layer (behaviors/): 2 backed by executable behavior

COVERED (SDD requirement <- executable behavior):
  cart#add-item  <-  behaviors/cart/cart.feature::添加商品
  cart#bulk-add  <-  behaviors/cart/cart.feature::批量添加

result: the SDD spec is fully backed by executable behavior.

How it works

1. Compile — each .feature becomes one test file under tests/behaviors/, each scenario one test function. New scenarios are appended; implemented tests are never touched (--force regenerates and discards implementations — only when you explicitly ask).
2. Anchor — every scenario gets a content hash written into the generated test as # bddfirst:anchor <hash>; pairings are recorded in .bddfirst/manifest.json.
3. Check — bddfirst check re-hashes both sides and diffs them: changed / added / removed scenarios, orphaned tests, duplicate anchors.
4. Audit — bddfirst audit extracts expected values (numbers, quoted strings) from Then steps and floor-checks that they appear in the test body.

Exit-code contract: 0 pass; 1 gate failed (drift / missing expected value); 2 precondition not ready (e.g. never built). CI treats only 1 as a gate signal.

OpenSpec integration

Keep using OpenSpec for requirements and SDD; bddfirst turns OpenSpec scenarios into executable behavior. Three levels:

1. Convention — a Gherkin scenario anchors to an OpenSpec requirement via a @spec(<capability>#<requirement>) tag; requirement names are slugified and normalized on both sides ("Rate Limiting" → rate-limiting).
2. Reconciliation — bddfirst coverage reads openspec/specs/ and reports COVERED / BEHAVIOR DEBT / UNTRACED / BROKEN LINK: is every SDD requirement backed by executable behavior?
3. Generation — bddfirst generate reads OpenSpec scenarios and generates Gherkin skeletons with the @spec tags already attached. --source changes (default) reads in-flight openspec/changes/ deltas; --source specs reads merged openspec/specs/.

bddfirst generate      # during a change: skeletons from the OpenSpec delta
bddfirst build         # compile into pytest skeletons
# ... the agent implements ...
bddfirst coverage      # after archive: reconcile specs/ against the behavior layer

Command reference

Command	Purpose
bddfirst init	One-shot setup: behaviors/ + Claude Code skill + AGENTS.md
bddfirst build [path]	Compile .feature → pytest skeletons (incremental, append-only by default)
bddfirst check	Scenario-level two-way drift check (gate)
bddfirst audit	Expected-value audit: Then-step values must appear in assertions (gate)
bddfirst list	Behavior map with implementation state (✓ implemented / ○ skeleton / ✗ missing)
bddfirst status	Pending backlog — promised-but-not-delivered scenarios with feature line numbers
bddfirst coverage	SDD ↔ BDD bridge: which OpenSpec requirements are backed by executable behavior
bddfirst generate	Generate behavior skeletons from OpenSpec scenarios

CI gate, three lines:

- run: bddfirst check
- run: bddfirst audit
- run: uv run pytest

(pre-commit can run the same fast pair bddfirst check && bddfirst audit; keep the full pytest in CI.)

Agent support

Agent	Mechanism
Claude Code	Project skill .claude/skills/bddfirst/SKILL.md (auto-triggered) + slash commands; both installed by bddfirst init, re-run to upgrade
Codex CLI / Copilot / Cursor / Jules / Amp / OpenCode / etc.	AGENTS.md (de-facto standard, one file covers the long tail)
Gemini CLI	Reads GEMINI.md: bridge with ln -s AGENTS.md GEMINI.md

Claude Code slash commands:

Command	What it does
/bddfirst:new <behavior description>	Write/modify the feature → build → report pending
/bddfirst:implement	Take the queue from status → fill skeletons → pytest green → audit
/bddfirst:gate	Commit gate: check + audit + pytest trio report

Capability boundary (important)

bddfirst is a floor-guarding tool, not a ceiling-guaranteeing tool. It keeps behavior from going astray; it cannot guarantee that the requirement itself is right, or that quality is high.

Guaranteed:

Guaranteed	Mechanism
Behavior matches the feature	check two-way drift check + executable tests
Expected values are not reverse-engineered from the implementation	audit expected-value audit
Specs and tests do not drift	scenario anchors + check

NOT guaranteed:

Not guaranteed	Why	Rely on
The feature reflects real human needs	spec-vs-human divergence is a human problem	human review of the feature
Edge cases were written, not just happy paths	audit only checks that expected values are asserted	human review + mutation testing
Architecture-level correctness	a test's scope is a module/component	architecture review + end-to-end tests

Higher coverage does not equal higher quality — with AI coding, coverage can go up while quality goes down. Executable tests are a necessary condition, not a sufficient one.

audit itself is a floor check, not a proof of correctness: an expected value counts once it appears in the test body, and always-true assertions still pass. It prevents "expected value absent or diverging from the feature"; semantic correctness still needs code review and mutation testing.

FAQ

• Difference from pytest-bdd? pytest-bdd needs a runtime + step definitions; bddfirst compiles native pytest files — zero runtime, zero step definitions.
• The feature changed — now what? bddfirst check reports exactly which scenario drifted; fix the tests per the report, or bddfirst build to append new skeletons.
• Windows? Not specifically adapted; expected to work in a UTF-8 terminal. The ✓○✗ marks degrade to [x]/[ ]/[!] in a GBK terminal.
• vitest / jest / go test? Not yet (driven by demand).
• How to exit? Remove behaviors/, tests/behaviors/, .bddfirst/, the skill dir, and uninstall the tool. Generated tests are plain pytest files and keep running.

License

MIT © cuikexi