Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,36 @@ project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
- **Hubs are now conformant Open Knowledge Format (OKF) concepts — "Surface = OKF + freshness."** A
hub's frontmatter is a superset of an [OKF](docs/guides/okf.md) concept: it carries OKF's `type`
(defaulted to `concept` so existing hubs are byte-unchanged and keep working), `title`, `tags`,
and `timestamp`, and **preserves any other frontmatter key verbatim** (OKF's `description`/
`resource`, a doc system's `author`/`created`/`pinned`) in an `extra` map — the OKF rule that
consumers must not drop unknown keys. So a hub drops into any OKF consumer (Google's Knowledge
Catalog, the OKF visualizer, Obsidian, git-backed editors), which read the prose and ignore the
`anchors:` Surface governs. `surf new` scaffolds `type: concept`.
- **Freshness provenance and stable claim IDs.** `surf verify` now records the freshness OKF omits:
a stable per-claim `id` (written once, never regenerated, so it survives prose/anchor edits — the
substrate for claim timelines) plus `verified_at` and `verified_commit`. The *who* is deliberately
not stored (git blame on the hub records it) — keeping author emails out of tracked files.
Provenance is written **only when the hash actually changes**, so a no-op re-verify stays
byte-identical. The `--format json` `Divergence` gains an additive `id` field (no report-version bump).
- **OKF bundle layout.** `surf.toml` gains a `bundles` glob: each root is a directory *tree* of
concept files, expanded as `<root>/**/*.md`. OKF reserved files (`index.md`, `log.md`) are
recognized and skipped for governance — they hold no claims and never block the gate.
- **`surf lint` OKF advisories.** A `Warn` (never a block) for an unknown frontmatter key that looks
like a typo of a known one (recovering the fail-closed signal below), a `Warn` when an anchored
hub has no headline (`summary`/`title`/`description`), and a `Warn` for a dangling OKF cross-link
in a hub body (OKF tolerates broken links, so this never blocks).

### Changed
- **Frontmatter no longer rejects unknown keys** (OKF requires consumers to preserve them). This is
a deliberate, narrow relaxation of the gate's fail-closed posture: a mistyped *frontmatter* key
that previously hard-blocked now parses and earns a `surf lint` warning instead. Unknown keys
**inside an anchor item** still fail closed — anchor items are Surface's own data, not OKF's.
- `summary` is now optional (an OKF concept may carry only `description`).

## [0.7.0] - 2026-06-29

### Added
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,15 @@ Surface's JSON output. The core never depends on it. More in
[What Surface does NOT do](docs/index.md#what-surface-does-not-do) and
[Is Surface for you?](docs/index.md#is-surface-for-you).

## Speaks OKF

A hub is a conformant [Open Knowledge Format](docs/guides/okf.md) concept — Google's vendor-neutral
standard for knowledge as markdown + frontmatter. OKF standardizes how knowledge is written down but
deliberately omits freshness; that's exactly what Surface adds. **Surface = OKF + the freshness OKF
leaves out.** Your hubs drop into any OKF consumer (Knowledge Catalog, the OKF visualizer, Obsidian,
git-backed doc editors), which read the prose and ignore the `anchors:` Surface governs. See
[Surface and OKF](docs/guides/okf.md).

## Install

Most repos never install the binary — they run the GitHub Action:
Expand Down Expand Up @@ -163,6 +172,7 @@ Full docs at **[surface.gradientdev.xyz](https://surface.gradientdev.xyz)** (sou

- [Quickstart](docs/getting-started/quickstart.md) · [Install](docs/getting-started/install.md)
- [Authoring hubs](docs/guides/authoring-hubs.md) — claims, anchor grammar, granularity, the verify loop.
- [Surface and OKF](docs/guides/okf.md) — hubs as conformant Open Knowledge Format concepts.
- [CI integration](docs/guides/ci-integration.md) — the Action, the pre-commit hook, scoping a PR.
- [Examples](docs/examples.md) — a minimal hub in each supported language.
- Reference: [Commands](docs/reference/commands.md) · [Configuration](docs/reference/configuration.md) · [How the gate works](docs/reference/how-it-works.md) · [FAQ](docs/reference/faq.md)
Expand Down
104 changes: 104 additions & 0 deletions docs/guides/okf.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
---
title: Surface and the Open Knowledge Format
description: A Surface hub is a conformant OKF concept; Surface adds the freshness OKF deliberately leaves out. Produce, consume, and govern OKF bundles.
---

Google's [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/tree/main/okf)
(OKF) is a vendor-neutral standard for representing knowledge as a directory of markdown files with
YAML frontmatter. It standardizes *how knowledge is written down* so a wiki produced by one tool can
be consumed by another without translation.

OKF is deliberately minimal. Its spec requires exactly one frontmatter field (`type`), recommends a
few more (`title`, `description`, `resource`, `tags`, `timestamp`), and mandates that consumers
**preserve unknown keys** and **never reject** a document for extra fields, unknown types, or broken
links. And it explicitly leaves several things undefined — most importantly, **freshness**: OKF has
no notion of "has the thing this describes changed?", no verification status, no anchoring to code,
no drift detection.

That gap is exactly what Surface fills.

## Surface = OKF + freshness

> A Surface hub is a **conformant OKF concept**. Surface adds the freshness layer OKF omits.

Because OKF ignores keys it doesn't recognize, Surface's governance metadata rides along inside an
otherwise-normal OKF concept:

```markdown
---
type: BigQuery Table # OKF: the one required field
title: Orders # OKF: display name
description: One row per order # OKF: preserved (Surface keeps it in `extra`)
tags: [sales, revenue] # OKF
timestamp: 2026-05-28T14:30:00Z # OKF: last *modified*
anchors: # Surface extension — OKF readers ignore it
- claim: an order is immutable once `status = shipped`
at: src/orders/model.ts > Order > freeze
hash: 2:9b1c33ade8f1
id: c_18be… # stable identity (claim timelines)
verified_at: 2026-06-30T09:12:00Z # last *attested* against the code
verified_commit: 4d5e6f2 # who is recovered from git blame, not stored
---

# Orders

Prose a human or agent reads to understand this table…
```

- **An OKF consumer** — Google's Knowledge Catalog, the OKF visualizer, [Nansidian](#doc-systems),
Obsidian — reads this as a normal concept and silently ignores `anchors`.
- **Surface** reads `anchors` and governs freshness: when `src/orders/model.ts > Order > freeze`
changes, [`surf check`](../reference/commands.md) fails until a human re-confirms the claim.
- The two timestamps are different on purpose: OKF's `timestamp` is *last modified*; Surface's
per-claim `verified_at` is *last attested against the code* — the freshness OKF can't express.

## What conformance means here

A hub is a conformant OKF concept when it carries a `type`. Surface makes this cheap:

- `surf new` scaffolds hubs with `type: concept` already set.
- Hubs written before OKF (no `type`) still parse — Surface treats a missing `type` as `concept`
in memory. They are byte-unchanged on disk; run a future `surf migrate` (or add `type:` by hand)
to make them OKF-conformant *on disk*.
- Any extra frontmatter key (OKF's `description`/`resource`, a doc system's `author`/`created`/
`pinned`) is preserved verbatim on round-trip — Surface never drops what it doesn't recognize.

The one boundary worth stating plainly:

> **Surface only fact-checks concepts that describe code.** A concept anchored to a code symbol is
> governed. A concept with no `anchors` (a BigQuery table's business meaning, an RFC, a playbook)
> is a valid, rendered, **ungoverned** OKF concept — it passes the gate untouched. Verifying
> non-code resources (e.g. a table's schema against the live warehouse) is future work; today the
> deterministic gate is scoped to code.

## Bundles

OKF ships knowledge as a **bundle**: a directory tree where each file is a concept, the path is its
identity, and two filenames are reserved — `index.md` (a directory listing for progressive
disclosure) and `log.md` (a change history). Point Surface at a bundle with `bundles` in
`surf.toml`:

```toml
# Govern a flat hubs/ dir, an in-repo OKF bundle, or both.
hubs = ["hubs/*.md"]
bundles = ["knowledge/sales"] # expands to knowledge/sales/**/*.md
```

Reserved files are recognized and skipped for governance (they hold no claims), so a bundle's
`index.md`/`log.md` never trip the gate. `surf lint` additionally checks OKF cross-links in a hub's
body and **warns** (never blocks — OKF tolerates broken links) on a dangling `.md` target.

## Doc systems

Because an OKF bundle is just markdown in a directory, it drops into any doc system that reads
markdown — [Obsidian](https://obsidian.md/) vaults, Notion imports, and git-backed editors. The
intended integration is a **CI gate on the git repos those systems sync**: run
[`surf check`](../guides/ci-integration.md) as a GitHub Action over the doc repo, so the freshness
gate guards the code-anchored subset of the knowledge base wherever the docs are edited.

## See also

- [Authoring hubs](./authoring-hubs.md) — a hub is an onboarding doc, not a claim-log (the same
shape OKF's prose-first concepts encourage).
- [Configuration](../reference/configuration.md) — the `bundles` glob and frontmatter fields.
- [How the gate works](../reference/how-it-works.md) — what Surface hashes and why.
12 changes: 11 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,16 @@ If you want the fuzzy "is this claim still true" judgment, that lives in an **op
plugin that reads Surface's JSON output. The core never depends on it. Pull every plugin out and
the gate blocks and passes exactly the same.

## Interoperable: Surface speaks OKF

A hub is a **conformant [Open Knowledge Format](./guides/okf.md) concept** — Google's vendor-neutral
standard for knowledge as markdown + frontmatter. OKF standardizes *how knowledge is written down*
but deliberately omits **freshness**: it has no notion of whether the thing a document describes has
changed. That omission is precisely what Surface is. So the relationship is clean: **Surface = OKF +
the freshness OKF leaves out.** Your hubs drop into any OKF consumer (Google's Knowledge Catalog,
the OKF visualizer, Obsidian, git-backed doc editors), which read the knowledge and ignore the
`anchors:` Surface governs. See [Surface and OKF](./guides/okf.md).

## Is Surface for you?

Honestly? Maybe not. Roughly, it earns its keep when
Expand All @@ -116,5 +126,5 @@ Agents are a multiplier, not the foundation.
## Next

- [Install](./getting-started/install.md) · [Quickstart](./getting-started/quickstart.md)
- [Authoring hubs](./guides/authoring-hubs.md) · [CI integration](./guides/ci-integration.md) · [Examples](./examples.md)
- [Authoring hubs](./guides/authoring-hubs.md) · [CI integration](./guides/ci-integration.md) · [Surface and OKF](./guides/okf.md) · [Examples](./examples.md)
- Reference: [Commands](./reference/commands.md) · [Configuration](./reference/configuration.md) · [How the gate works](./reference/how-it-works.md) · [Hash recipes](./reference/hash-recipes.md) · [FAQ](./reference/faq.md)
36 changes: 36 additions & 0 deletions docs/reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,42 @@ layout and the trade-offs).
> derived from the first pattern (e.g. `docs/hubs/*.md` → `docs/hubs/`); the other patterns are
> still linted and verified normally, they're just not where `new` writes.

## OKF bundles

To govern an [Open Knowledge Format](../guides/okf.md) bundle — a directory *tree* of concept files
rather than a flat folder — list its root(s) under `bundles`. Each root expands to `<root>/**/*.md`:

```toml
hubs = ["hubs/*.md"] # optional; flat layout
bundles = ["knowledge/sales"] # an in-repo OKF bundle (recursively)
```

`hubs` and `bundles` are unioned. OKF reserved files (`index.md`, `log.md`) swept up by a bundle
glob are recognized and **skipped for governance** — they hold no claims, so a missing frontmatter
fence in them never blocks the gate.

## Frontmatter fields

A hub is a **superset of an OKF concept**, so its frontmatter carries both OKF fields and Surface's
governance fields:

| Field | Source | Notes |
| --- | --- | --- |
| `type` | OKF | The one field OKF requires. Defaults to `concept` when absent (existing hubs keep working). |
| `title` | OKF | Display name. |
| `summary` | Surface | The onboarding one-liner (optional). Distinct from OKF `description`. |
| `tags` | OKF | Cross-cutting tags. |
| `timestamp` | OKF | Last *modified* (ISO 8601). Distinct from a claim's `verified_at` (last *attested*). |
| `anchors` | Surface | The claims — the governed part. See [Authoring hubs](../guides/authoring-hubs.md). |
| `refs` / `covers` | Surface | Composition edges and advisory coverage globs. |
| *(any other key)* | OKF / tools | Preserved verbatim (e.g. OKF `description`/`resource`, a doc system's `author`/`created`). |

Unknown *frontmatter* keys are preserved, not rejected (the OKF rule); a key that looks like a typo
of a known one earns a `surf lint` warning. Unknown keys **inside an anchor item** still fail
closed. Each claim also gains freshness provenance the first time `surf verify` stamps it: a stable
`id`, plus `verified_at` and `verified_commit` (the *who* is left to git blame, so no author email
lands in tracked files).

## Languages

TypeScript (`.ts`, `.tsx`, `.mts`, `.cts`), JavaScript/JSX (`.js`, `.jsx`, `.mjs`, `.cjs`), Rust
Expand Down
39 changes: 24 additions & 15 deletions hubs/cli-check.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@ anchors:
a mismatch → Changed; a clean match is tagged with whether the stamp was still v1. The
verdict is deterministic and needs no git.
at: surf-cli/src/check.rs > check_claim
hash: 2:66e7b4149d60
hash: 2:7e984c24d736
id: c_18be38a62d95d1d80001
verified_at: 2026-07-01T16:53:08Z
verified_commit: 7c5aabe74da3b56ff680044aeb3b20747b606479
- claim: >
Scoping is opt-in and intersective: with neither --base nor --files every claim is checked.
A claim is in scope when any of its anchored files matches each active filter — the --base
Expand All @@ -19,17 +22,22 @@ anchors:
at: surf-cli/src/check.rs > Scope > includes
hash: 2:64277175938c
- claim: >
The gate fails closed: a hub whose frontmatter won't parse yields an Unresolvable
divergence (blocking the run) rather than being silently skipped, so a frontmatter typo
can't pass as clean. After the per-claim walk it propagates refs one hop — a hub that
directly references a stale hub (or a stale claim within one) inherits a ReferencedStale
divergence, built only from base divergences so a chain stops at the first hop. Alongside
the divergences it returns the --files patterns that matched no anchored file (run warns on
stderr for each and exits non-zero when every pattern matched nothing, so a typo'd --files
can't read as a clean run) and a count of clean anchors still stamped under v1, so run can
nudge the one-time `surf verify` upgrade.
The gate fails closed for concepts: a concept hub whose frontmatter won't parse yields an
Unresolvable divergence (blocking the run) rather than being silently skipped, so a
frontmatter typo can't pass as clean. OKF reserved files (index.md/log.md) are the exception
— they carry no claims, so they are skipped entirely and never block even without frontmatter.
After the per-claim walk it propagates refs one hop — a hub that directly references a stale
hub (or a stale claim within one) inherits a ReferencedStale divergence, built only from base
divergences so a chain stops at the first hop. Alongside the divergences it returns the
--files patterns that matched no anchored file (run warns on stderr for each and exits
non-zero when every pattern matched nothing, so a typo'd --files can't read as a clean run)
and a count of clean anchors still stamped under v1, so run can nudge the one-time
`surf verify` upgrade.
at: surf-cli/src/check.rs > check_workspace
hash: 2:b7b7fd55206e
hash: 2:bed6c18b3cc1
id: c_18be38a62f7582a00002
verified_at: 2026-07-01T16:53:08Z
verified_commit: 7c5aabe74da3b56ff680044aeb3b20747b606479
refs:
- ./cli-git.md
- ./cli-verify.md
Expand All @@ -44,10 +52,11 @@ produces the same answer; the git helpers in [`cli-git.md`](./cli-git.md) only f

`check_claim` is the per-claim verdict; `check_workspace` walks every hub, and `Scope` narrows
which claims it evaluates when `--base` or `--files` is given — opt-in and intersective, falling
back to a full check rather than checking nothing. Any divergence (including a hub whose
frontmatter won't parse — the gate fails closed) makes `run` exit non-zero. A hub also fails when a
hub it [`refs`](./hub-format.md) is stale: composition propagates one hop (#4), so the gate that
flags a dependency flags everything built on it.
back to a full check rather than checking nothing. Any divergence (including a *concept* hub whose
frontmatter won't parse — the gate fails closed) makes `run` exit non-zero. OKF reserved files
(`index.md`/`log.md`) hold no claims, so they are skipped rather than governed. A hub also fails
when a hub it [`refs`](./hub-format.md) is stale: composition propagates one hop (#4), so the gate
that flags a dependency flags everything built on it.

**Boundary:** green means "nothing anchored changed since last sign-off," not "the prose is true";
that confirmation is [`surf verify`](./cli-verify.md)'s job, not the gate's.
15 changes: 10 additions & 5 deletions hubs/cli-workspace.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,13 @@ anchors:
at: surf-cli/src/workspace.rs > Workspace > discover
hash: 2:7d57c89fcc0d
- claim: >
hub_paths globs the config's hub patterns relative to the discovered root, sorted and
deduped.
hub_paths globs the config's hub patterns relative to the discovered root, then expands each
OKF bundle root as `<root>/**/*.md`, returning the combined set sorted and deduped.
at: surf-cli/src/workspace.rs > Workspace > hub_paths
hash: 2:c69c8264bcfd
hash: 2:0e986d323b98
id: c_18be38a6357318480003
verified_at: 2026-07-01T16:53:08Z
verified_commit: 7c5aabe74da3b56ff680044aeb3b20747b606479
refs:
- ./cli-check.md
- ./cli-lint.md
Expand All @@ -26,8 +29,10 @@ resolved root rather than the caller's current directory.
`discover` is what makes `surf` runnable from any subdirectory — it walks up to the nearest
`surf.toml` (the same root-finding git and ruff use) and errors if none is found, so a stray
invocation outside a project fails loudly instead of silently governing nothing. The resolved root
is the base every anchor path is joined against, and `hub_paths` globs the configured patterns
relative to it (sorted and deduped) to enumerate the hubs.
is the base every anchor path is joined against, and `hub_paths` enumerates the hubs by globbing the
configured `hubs` patterns and expanding any OKF `bundles` roots (each as `<root>/**/*.md`), sorted
and deduped. Reserved OKF files swept up this way are classified on `LoadedHub` and skipped by the
governing commands.

**Boundary:** discovery and enumeration only — it parses no hub bodies and resolves no anchors;
that is [`lint`](./cli-lint.md)/[`check`](./cli-check.md)'s job over the files this hands back.
Loading