Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@
# DB_DRIVER=postgres
# DATABASE_URL=postgres://user:pass@host:5432/dbname

# --- webhooks (outbound; deliveries are persisted + retried) ---
# WEBHOOK_MAX_ATTEMPTS=5 # attempts before a delivery is marked failed
# WEBHOOK_TIMEOUT_MS=10000 # per-delivery HTTP timeout

# --- realtime: in-memory (default) | redis ---
# Unset = single-process in-memory pub/sub + presence (the zero-config default).
# Set REDIS_URL to scale horizontally across multiple server processes: websocket
Expand Down
38 changes: 38 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ That's it — no clone, no config, no database to provision. Open
| **Files** | Upload / download / list / delete — local disk by default, **S3** (or any S3-compatible store) optional |
| **AI** | Chat (JSON or SSE streaming) over a **pluggable provider** — Anthropic, any OpenAI-compatible endpoint (OpenAI, **LiteLLM**, Ollama…), or **Amazon Bedrock** |
| **Realtime** | Websocket pub/sub with presence (join/leave/typing + roster), history, and an optional **Redis** fan-out for multi-process scale |
| **Webhooks** | Outbound HTTP on data/file events — HMAC-signed, with a **persisted, retried** delivery queue and delivery history |
| **Multi-tenant** | Optional isolated projects — one VM hosts many sites (set `ADMIN_KEY`) |
| **Backup** | Whole-instance export / restore as one JSON file |

Expand Down Expand Up @@ -354,6 +355,43 @@ In multi-tenant mode, the live feed of a `read:public` collection
project id** (`{ project }`) — a read-only subscription, so a published frontend
gets live updates without a secret. Other channels still require a project key.

### Webhooks — outbound events

Register a URL to receive `data.*` / `file.*` events as HTTP POSTs. Deliveries are
**persisted and retried** with exponential backoff (so a brief outage on your side
doesn't drop events), and you can inspect delivery history.

```bash
# Subscribe to created/updated docs in the "orders" collection
curl -X POST localhost:3737/v1/webhooks -H 'content-type: application/json' -d '{
"url": "https://example.com/hook",
"events": ["data.created", "data.updated"],
"collections": ["orders"]
}' # -> { id, url, events, secret: "whsec_…", ... } (secret shown once)

curl localhost:3737/v1/webhooks # list (without secrets)
curl localhost:3737/v1/webhooks/<id>/deliveries # recent attempts + status
curl -X POST localhost:3737/v1/webhooks/<id>/rotate-secret
curl -X DELETE localhost:3737/v1/webhooks/<id>
```

Event types use a `resource.action` shape (`data.created`, `data.updated`,
`data.deleted`, `file.created`, `file.deleted`); subscribe to exact types, a prefix
(`data.*`), or all (`*`), and optionally filter by `collections`. Each delivery is a
JSON POST `{ id, event, createdAt, project, data }` with headers:

| Header | Meaning |
| --- | --- |
| `X-Zero-Event` | the event type |
| `X-Zero-Delivery` | unique delivery id |
| `X-Zero-Timestamp` | send time (ms) |
| `X-Zero-Signature` | `sha256=<hex>` — HMAC-SHA256 of `"<timestamp>.<rawBody>"` keyed by the hook's `secret` |

Verify by recomputing the HMAC over `` `${timestamp}.${rawBody}` `` with your stored
secret and comparing to `X-Zero-Signature`. Management is key-gated (and per-project
in multi-tenant); since webhooks make outbound requests, run with a key and restrict
egress if the instance is internet-exposed.

## Multi-tenant projects

By default the server runs in **open mode**: one implicit project, no auth — exactly
Expand Down
17 changes: 17 additions & 0 deletions public/docs.html
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ <h1 style="margin: 0 0 0.4rem">Docs</h1>
<a href="#rest-files">REST · Files</a>
<a href="#rest-ai">REST · AI</a>
<a href="#rest-realtime">REST · Realtime</a>
<a href="#rest-webhooks">REST · Webhooks</a>
<a href="#rest-admin">REST · Admin / backup</a>
<a href="#rest-meta">REST · Meta &amp; health</a>
<a href="#queries">Queries &amp; filters</a>
Expand Down Expand Up @@ -276,6 +277,22 @@ <h2 id="rest-realtime">REST API — Realtime</h2>
<span class="c"># Zero(base, { project: 'prj_…' }).data('posts').subscribe(fn) // read-only</span>
<span class="c"># Scale out: set REDIS_URL and run many processes — frames + presence fan out via Redis.</span></pre>

<h2 id="rest-webhooks">REST API — Webhooks</h2>
<table>
<tbody>
<tr><td class="ep"><span class="m">POST</span>/v1/webhooks</td><td>Register <code>{ url, events:[…], collections?:[…], secret?, active? }</code> → returns the hook + signing <code>secret</code> (shown here only).</td></tr>
<tr><td class="ep"><span class="m">GET</span>/v1/webhooks</td><td>List hooks (secrets omitted).</td></tr>
<tr><td class="ep"><span class="m">GET·PATCH·DELETE</span>/v1/webhooks/:id</td><td>Read (with secret) / update url·events·collections·active / delete.</td></tr>
<tr><td class="ep"><span class="m">POST</span>/v1/webhooks/:id/rotate-secret</td><td>Issue a new signing secret.</td></tr>
<tr><td class="ep"><span class="m">GET</span>/v1/webhooks/:id/deliveries</td><td>Recent delivery attempts: status, attempts, last error/status code.</td></tr>
</tbody>
</table>
<p class="note">Events: <code>data.created</code> · <code>data.updated</code> · <code>data.deleted</code> · <code>file.created</code> · <code>file.deleted</code>. Subscribe to exact types, a prefix (<code>data.*</code>), or all (<code>*</code>). Deliveries are <b>persisted and retried</b> with backoff.</p>
<pre class="code"><span class="c"># Each delivery POSTs { id, event, createdAt, project, data } with headers:</span>
<span class="c"># X-Zero-Event, X-Zero-Delivery, X-Zero-Timestamp, X-Zero-Signature: sha256=&lt;hex&gt;</span>
<span class="c"># Verify: hmacSHA256(secret, `${X-Zero-Timestamp}.${rawBody}`) === signature</span>
<span class="k">const</span> sig = crypto.createHmac(<span class="grad">'sha256'</span>, secret).update(ts + <span class="grad">'.'</span> + rawBody).digest(<span class="grad">'hex'</span>);</pre>

<h2 id="rest-admin">REST API — Admin &amp; backup</h2>
<table>
<tbody>
Expand Down
35 changes: 35 additions & 0 deletions public/sdk.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,40 @@ declare namespace Zero {
ai: { provider: string; ready: boolean; model: string | null };
}

interface Webhook {
id: string;
url: string;
events: string[];
collections: string[];
active: boolean;
createdAt: number;
/** Returned on create/get/rotate-secret only — used to verify signatures. */
secret?: string;
}

interface WebhookDelivery {
id: string;
event: string;
status: 'pending' | 'delivering' | 'delivered' | 'failed';
attempts: number;
maxAttempts: number;
lastError: string | null;
lastStatusCode: number | null;
nextAttemptAt: number;
createdAt: number;
updatedAt: number;
}

interface Webhooks {
create(config: { url: string; events: string[]; collections?: string[]; secret?: string; active?: boolean }): Promise<Webhook>;
list(): Promise<Webhook[]>;
get(id: string): Promise<Webhook>;
update(id: string, patch: { url?: string; events?: string[]; collections?: string[]; active?: boolean }): Promise<Webhook>;
rotateSecret(id: string): Promise<Webhook>;
remove(id: string): Promise<{ deleted: boolean }>;
deliveries(id: string): Promise<WebhookDelivery[]>;
}

interface Client {
baseUrl: string;
health(): Promise<{ ok: boolean; uptime: number }>;
Expand All @@ -248,5 +282,6 @@ declare namespace Zero {
files: Files;
ai: AI;
realtime(channel: string, opts?: RoomOptions): Room;
webhooks: Webhooks;
}
}
15 changes: 15 additions & 0 deletions public/sdk.js
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,20 @@
};
}

// Outbound webhooks (operator/tenant scope). create() returns the signing
// secret; deliveries() shows recent attempts + status.
var webhooks = {
create: function (config) { return request('POST', '/v1/webhooks', config || {}); },
list: function () { return request('GET', '/v1/webhooks').then(function (r) { return r.webhooks; }); },
get: function (id) { return request('GET', '/v1/webhooks/' + encodeURIComponent(id)); },
update: function (id, patch) { return request('PATCH', '/v1/webhooks/' + encodeURIComponent(id), patch || {}); },
rotateSecret: function (id) { return request('POST', '/v1/webhooks/' + encodeURIComponent(id) + '/rotate-secret'); },
remove: function (id) { return request('DELETE', '/v1/webhooks/' + encodeURIComponent(id)); },
deliveries: function (id) {
return request('GET', '/v1/webhooks/' + encodeURIComponent(id) + '/deliveries').then(function (r) { return r.deliveries; });
},
};

return {
baseUrl: base,
health: function () { return request('GET', '/health'); },
Expand All @@ -377,6 +391,7 @@
files: files,
ai: ai,
realtime: realtime,
webhooks: webhooks,
};
}

Expand Down
6 changes: 6 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,12 @@ export const config = {
redisUrl: process.env.REDIS_URL ?? null,
},

// Outbound webhooks: deliveries are persisted and retried with backoff.
webhooks: {
maxAttempts: Number(process.env.WEBHOOK_MAX_ATTEMPTS ?? 5),
timeoutMs: Number(process.env.WEBHOOK_TIMEOUT_MS ?? 10_000),
},

ai: {
// Which backend serves /v1/ai/chat: anthropic (default) | openai | bedrock.
// - anthropic: Anthropic Messages API (ANTHROPIC_API_KEY).
Expand Down
32 changes: 32 additions & 0 deletions src/db/postgres.ts
Original file line number Diff line number Diff line change
Expand Up @@ -177,5 +177,37 @@ async function bootstrap(pool: pg.Pool): Promise<void> {
schema text NOT NULL,
PRIMARY KEY (project, collection)
);

CREATE TABLE IF NOT EXISTS webhooks (
id text PRIMARY KEY,
project text NOT NULL DEFAULT 'default',
url text NOT NULL,
events text NOT NULL,
collections text NOT NULL DEFAULT '[]',
secret text NOT NULL,
active boolean NOT NULL DEFAULT true,
created_at bigint NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_webhooks_project ON webhooks (project);

CREATE TABLE IF NOT EXISTS webhook_deliveries (
id text PRIMARY KEY,
webhook_id text NOT NULL,
project text NOT NULL,
event text NOT NULL,
url text NOT NULL,
secret text NOT NULL,
payload text NOT NULL,
status text NOT NULL DEFAULT 'pending',
attempts integer NOT NULL DEFAULT 0,
max_attempts integer NOT NULL,
next_attempt_at bigint NOT NULL,
last_error text,
last_status_code integer,
created_at bigint NOT NULL,
updated_at bigint NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_wh_deliveries_due ON webhook_deliveries (status, next_attempt_at);
CREATE INDEX IF NOT EXISTS idx_wh_deliveries_hook ON webhook_deliveries (webhook_id, created_at DESC);
`);
}
35 changes: 35 additions & 0 deletions src/db/sqlite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,41 @@ export function bootstrap(db: Database.Database): void {
);
`);

// --- outbound webhooks + their persisted, retried delivery queue ---
db.exec(`
CREATE TABLE IF NOT EXISTS webhooks (
id TEXT PRIMARY KEY,
project TEXT NOT NULL DEFAULT 'default',
url TEXT NOT NULL,
events TEXT NOT NULL, -- JSON array of event patterns
collections TEXT NOT NULL DEFAULT '[]', -- JSON array; empty = all
secret TEXT NOT NULL,
active INTEGER NOT NULL DEFAULT 1,
created_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_webhooks_project ON webhooks (project);

CREATE TABLE IF NOT EXISTS webhook_deliveries (
id TEXT PRIMARY KEY,
webhook_id TEXT NOT NULL,
project TEXT NOT NULL,
event TEXT NOT NULL,
url TEXT NOT NULL, -- snapshot, so a deleted/edited hook can't orphan it
secret TEXT NOT NULL,
payload TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'pending', -- pending|delivering|delivered|failed
attempts INTEGER NOT NULL DEFAULT 0,
max_attempts INTEGER NOT NULL,
next_attempt_at INTEGER NOT NULL,
last_error TEXT,
last_status_code INTEGER,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_wh_deliveries_due ON webhook_deliveries (status, next_attempt_at);
CREATE INDEX IF NOT EXISTS idx_wh_deliveries_hook ON webhook_deliveries (webhook_id, created_at DESC);
`);

// --- full-text index (rebuilt if it predates the project column) ---
if (tableExists(db, 'documents_fts') && !hasColumn(db, 'documents_fts', 'project')) {
db.exec(`DROP TABLE documents_fts;`);
Expand Down
29 changes: 23 additions & 6 deletions src/modules/data/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,25 @@ export async function dataRoutes(app: FastifyInstance): Promise<void> {

// Broadcast a change so clients subscribed to `data:<collection>` get live
// updates. Channels are namespaced per project by the realtime routes.
const emit = (project: string, collection: string, event: Record<string, unknown>) =>
// Broadcast a change over realtime AND durably enqueue it for matching webhooks
// (best-effort: a webhook failure never blocks the write). data.created/updated
// carry the document; data.deleted carries just the id.
const emit = async (
project: string,
collection: string,
event: { type: 'created' | 'updated' | 'deleted'; collection: string; id: string; document?: unknown },
): Promise<void> => {
app.realtime.publish(`${project}:data:${collection}`, event, { history: false });
try {
await app.webhooks.enqueue(project, `data.${event.type}`, {
collection,
id: event.id,
...(event.document !== undefined ? { document: event.document } : {}),
});
} catch (err) {
app.log.error({ err }, 'webhook enqueue failed');
}
};

// For an anonymous (public-ACL) request, the collection's hidden fields to
// strip from read results; empty for authenticated callers (they see all).
Expand Down Expand Up @@ -138,7 +155,7 @@ export async function dataRoutes(app: FastifyInstance): Promise<void> {
const { collection } = req.params as { collection: string };
const body = (req.body ?? {}) as Record<string, unknown>;
const doc = await store.create(req.projectId, collection, body);
emit(req.projectId, collection, { type: 'created', collection, id: doc.id, document: doc });
await emit(req.projectId, collection, { type: 'created', collection, id: doc.id, document: doc });
reply.code(201);
return doc;
});
Expand Down Expand Up @@ -173,7 +190,7 @@ export async function dataRoutes(app: FastifyInstance): Promise<void> {

const created = await store.createMany(req.projectId, collection, items as Record<string, unknown>[]);
for (const doc of created) {
emit(req.projectId, collection, { type: 'created', collection, id: doc.id, document: doc });
await emit(req.projectId, collection, { type: 'created', collection, id: doc.id, document: doc });
}
reply.code(201);
return { created };
Expand Down Expand Up @@ -341,7 +358,7 @@ export async function dataRoutes(app: FastifyInstance): Promise<void> {
const doc = await write((req.body ?? {}) as Record<string, unknown>, ifMatchVersion(req));
if (!doc) return reply.code(404).send({ error: 'not_found' });
reply.header('ETag', String(doc.updatedAt));
emit(req.projectId, collection, { type: 'updated', collection, id: doc.id, document: doc });
await emit(req.projectId, collection, { type: 'updated', collection, id: doc.id, document: doc });
return doc;
} catch (err) {
if (err instanceof VersionConflictError) {
Expand Down Expand Up @@ -386,14 +403,14 @@ export async function dataRoutes(app: FastifyInstance): Promise<void> {
});
}
const ids = await store.deleteMany(req.projectId, collection, filters, orFilters);
for (const id of ids) emit(req.projectId, collection, { type: 'deleted', collection, id });
for (const id of ids) await emit(req.projectId, collection, { type: 'deleted', collection, id });
return { deleted: ids.length };
});

app.delete('/v1/data/:collection/:id', async (req, reply) => {
const { collection, id } = req.params as { collection: string; id: string };
if (!(await store.delete(req.projectId, collection, id))) return reply.code(404).send({ error: 'not_found' });
emit(req.projectId, collection, { type: 'deleted', collection, id });
await emit(req.projectId, collection, { type: 'deleted', collection, id });
return { deleted: true };
});
}
16 changes: 16 additions & 0 deletions src/modules/files/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,15 @@ import { FileStore } from './storage.js';
export async function fileRoutes(app: FastifyInstance): Promise<void> {
const store = new FileStore(app.db, app.blobs);

// Durably enqueue a file event for matching webhooks (best-effort).
const fire = async (project: string, event: string, data: Record<string, unknown>): Promise<void> => {
try {
await app.webhooks.enqueue(project, event, data);
} catch (err) {
app.log.error({ err }, 'webhook enqueue failed');
}
};

app.post('/v1/files', async (req, reply) => {
const data = await req.file();
if (!data) {
Expand All @@ -25,6 +34,12 @@ export async function fileRoutes(app: FastifyInstance): Promise<void> {
return reply.code(413).send({ error: 'file_too_large' });
}

await fire(req.projectId, 'file.created', {
id: meta.id,
name: meta.name,
contentType: meta.contentType,
size: meta.size,
});
reply.code(201);
return meta;
});
Expand Down Expand Up @@ -53,6 +68,7 @@ export async function fileRoutes(app: FastifyInstance): Promise<void> {
app.delete('/v1/files/:id', async (req, reply) => {
const { id } = req.params as { id: string };
if (!(await store.delete(req.projectId, id))) return reply.code(404).send({ error: 'not_found' });
await fire(req.projectId, 'file.deleted', { id });
return { deleted: true };
});
}
6 changes: 6 additions & 0 deletions src/modules/meta/routes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,12 @@ export async function metaRoutes(app: FastifyInstance): Promise<void> {
provider: config.ai.provider, // anthropic | openai | bedrock (AI_PROVIDER)
chat: 'POST /v1/ai/chat { messages, system?, model?, max_tokens?, stream? }',
},
webhooks: {
manage: 'POST/GET /v1/webhooks · GET/PATCH/DELETE /v1/webhooks/:id · POST /v1/webhooks/:id/rotate-secret (key-gated)',
deliveries: 'GET /v1/webhooks/:id/deliveries — recent delivery attempts + status',
events: ['data.created', 'data.updated', 'data.deleted', 'file.created', 'file.deleted'],
signing: 'HMAC-SHA256 over "<X-Zero-Timestamp>.<body>" in X-Zero-Signature: sha256=<hex>',
},
realtime: {
subscribe: 'WS /v1/realtime/:channel?as=<identity> (frames: type=message|presence; presence events: sync/join/leave/typing)',
publish: 'POST /v1/realtime/:channel',
Expand Down
Loading
Loading