AWA can run with a single database user, but production deployments should separate schema management from runtime execution. This guide documents the minimum-privilege role model.
awa_owner NOLOGIN — owns all schema objects
├── awa_migrator LOGIN — runs migrations (inherits awa_owner)
└── awa_runtime LOGIN — workers, producers, awa serve, CLI admin
awa_owner is a NOLOGIN group role that owns the awa schema and all objects in it. No process connects as awa_owner directly.
awa_migrator is a LOGIN role that is a member of awa_owner. It runs awa migrate and can create/alter/drop schema objects. Use this role only for migrations — not for workers or the UI.
awa_runtime is a LOGIN role with the minimum privileges needed to run workers, enqueue jobs, and serve the admin UI. It cannot modify the schema.
-- Run as a superuser or database owner
CREATE ROLE awa_owner NOLOGIN;
CREATE ROLE awa_migrator LOGIN PASSWORD 'strong-password-here';
CREATE ROLE awa_runtime LOGIN PASSWORD 'strong-password-here';
-- awa_migrator inherits awa_owner (can create/alter schema objects)
GRANT awa_owner TO awa_migrator;
-- Both roles need to connect
GRANT CONNECT ON DATABASE mydb TO awa_migrator;
GRANT CONNECT ON DATABASE mydb TO awa_runtime;
-- awa_owner needs CREATE to make the schema
GRANT CREATE ON DATABASE mydb TO awa_owner;awa --database-url "postgres://awa_migrator:pass@host/mydb" migrateThe migrator creates the awa schema and all objects. Objects are owned by awa_migrator (who inherits awa_owner).
After the initial migration, transfer object ownership to awa_owner so it's decoupled from the login role:
ALTER SCHEMA awa OWNER TO awa_owner;
-- Transfer tables, partitioned tables, views, materialized views, and sequences.
DO $$
DECLARE r RECORD;
BEGIN
FOR r IN
SELECT c.relkind, c.oid::regclass AS obj
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'awa'
AND c.relkind IN ('r', 'p', 'v', 'm', 'S')
LOOP
IF r.relkind = 'S' THEN
EXECUTE format('ALTER SEQUENCE %s OWNER TO awa_owner', r.obj);
ELSE
EXECUTE format('ALTER TABLE %s OWNER TO awa_owner', r.obj);
END IF;
END LOOP;
END$$;
-- Transfer functions.
DO $$
DECLARE r RECORD;
BEGIN
FOR r IN SELECT p.oid::regprocedure AS func
FROM pg_proc p JOIN pg_namespace n ON p.pronamespace = n.oid
WHERE n.nspname = 'awa' LOOP
EXECUTE format('ALTER FUNCTION %s OWNER TO awa_owner', r.func);
END LOOP;
END$$;
-- Transfer standalone enum/domain types. Table row types and generated array
-- types are owned through their base objects and should not be altered here.
DO $$
DECLARE r RECORD;
BEGIN
FOR r IN
SELECT format('%I.%I', n.nspname, t.typname) AS typ
FROM pg_type t
JOIN pg_namespace n ON n.oid = t.typnamespace
WHERE n.nspname = 'awa'
AND t.typtype IN ('d', 'e')
LOOP
EXECUTE format('ALTER TYPE %s OWNER TO awa_owner', r.typ);
END LOOP;
END$$;-- Schema access
GRANT USAGE ON SCHEMA awa TO awa_runtime;
-- Sequences: canonical `jobs_id_seq`, and queue-storage `job_id_seq`
-- once prepare_schema has materialized it.
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA awa TO awa_runtime;
-- All tables: the runtime needs full DML because triggers run as the
-- invoking role (SECURITY INVOKER), so inserting a job also writes to
-- the admin metadata cache tables via triggers. The maintenance leader
-- also calls refresh_admin_metadata(), which truncates dirty-key tables
-- after taking the metadata advisory lock.
GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON ALL TABLES IN SCHEMA awa TO awa_runtime;
-- Functions (trigger functions execute with invoker privileges)
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA awa TO awa_runtime;
-- Keep the queue-storage installer helper migrator/operator-only.
REVOKE EXECUTE ON FUNCTION awa.install_queue_storage_substrate(TEXT, INT, INT, INT, BOOLEAN)
FROM awa_runtime;
-- Default privileges for future migrations.
ALTER DEFAULT PRIVILEGES FOR ROLE awa_owner IN SCHEMA awa
GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON TABLES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_owner IN SCHEMA awa
GRANT USAGE, SELECT ON SEQUENCES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_owner IN SCHEMA awa
GRANT EXECUTE ON FUNCTIONS TO awa_runtime;
-- If migrations run as awa_migrator without `SET ROLE awa_owner`, future
-- objects are owned by awa_migrator, so set defaults for that role too.
ALTER DEFAULT PRIVILEGES FOR ROLE awa_migrator IN SCHEMA awa
GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON TABLES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_migrator IN SCHEMA awa
GRANT USAGE, SELECT ON SEQUENCES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_migrator IN SCHEMA awa
GRANT EXECUTE ON FUNCTIONS TO awa_runtime;If you use the compatibility insert_many_copy path (Rust InsertOpts::copy()), also grant:
GRANT TEMP ON DATABASE mydb TO awa_runtime;This allows creating temporary staging tables for the COPY bulk insert path. Queue-storage direct COPY (QueueStorage::enqueue_params_copy in Rust, enqueue_many_copy in Python) writes to the queue-storage tables directly and does not need this temporary-table grant.
The queue-storage backend defaults to keeping its tables in the same awa schema, so the grants above cover both control-plane and queue-storage tables. See Queue-storage substrate for the full ownership contract — what awa migrate installs by default, how custom schemas are materialised via awa.install_queue_storage_substrate(), and why awa storage prepare-queue-storage-schema --schema awa --reset is rejected.
If you override the schema name (Rust: QueueStorageConfig.schema; Python: queue_storage_schema=...; CLI: awa storage prepare-queue-storage-schema --schema <name>), repeat the grant block against that schema:
GRANT USAGE ON SCHEMA my_qs_schema TO awa_runtime;
GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON ALL TABLES IN SCHEMA my_qs_schema TO awa_runtime;
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA my_qs_schema TO awa_runtime;
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA my_qs_schema TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_owner IN SCHEMA my_qs_schema
GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON TABLES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_owner IN SCHEMA my_qs_schema
GRANT USAGE, SELECT ON SEQUENCES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_owner IN SCHEMA my_qs_schema
GRANT EXECUTE ON FUNCTIONS TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_migrator IN SCHEMA my_qs_schema
GRANT SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON TABLES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_migrator IN SCHEMA my_qs_schema
GRANT USAGE, SELECT ON SEQUENCES TO awa_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE awa_migrator IN SCHEMA my_qs_schema
GRANT EXECUTE ON FUNCTIONS TO awa_runtime;The default awa queue-storage substrate is migrated by awa migrate. Custom queue-storage schemas are migrated by awa storage prepare-queue-storage-schema, which should also run as the migrator role and creates objects owned by awa_migrator / awa_owner. The runtime role needs read/write/execute privileges plus TRUNCATE for ring-partition rotation; it never needs DDL.
| Process | Role | Connection string |
|---|---|---|
awa migrate |
awa_migrator |
postgres://awa_migrator:pass@host/db |
| Workers (Rust/Python) | awa_runtime |
postgres://awa_runtime:pass@host/db |
awa serve (UI + API) |
awa_runtime |
postgres://awa_runtime:pass@host/db |
awa job, awa queue, etc |
awa_runtime |
postgres://awa_runtime:pass@host/db |
The runtime grants look broad (SELECT, INSERT, UPDATE, DELETE, TRUNCATE ON ALL TABLES) because AWA's internal tables are maintained by SECURITY INVOKER trigger functions. When a worker inserts a job, triggers automatically update:
awa.queue_state_counts— per-queue job state talliesawa.job_kind_catalog— distinct job kindsawa.job_queue_catalog— distinct queuesawa.job_unique_claims— unique key deduplication
Since these triggers run with the caller's privileges, the runtime role needs write access to these internal tables even though application code never touches them directly.
The maintenance leader also calls awa.refresh_admin_metadata() as a full reconciliation safety net. That function runs with invoker privileges and uses TRUNCATE on awa.admin_dirty_queues and awa.admin_dirty_kinds after taking the metadata advisory lock, so awa_runtime needs the TRUNCATE table privilege too.
The runtime also directly upserts into the descriptor catalogs on startup and on each runtime snapshot tick:
awa.queue_descriptors— declared queue display names, ownership, tagsawa.job_kind_descriptors— declared job-kind display names, ownership, tagsawa.runtime_instances— the runtime's own liveness row, including per-queue and per-kind descriptor hashes used for drift detection
These writes are not trigger-driven; they come from ClientBuilder::build() / AsyncClient.start() and from the snapshot reporter. The broad grant already covers them.
Other PostgreSQL features used at runtime:
| Feature | Purpose | Privilege needed |
|---|---|---|
LISTEN / NOTIFY |
Queue wakeup without polling | CONNECT (no extra grant) |
pg_try_advisory_lock |
Leader election for maintenance | Built-in function (no grant) |
COPY ... FROM STDIN |
Bulk insert path | TEMP on database |
FOR UPDATE SKIP LOCKED |
Non-blocking job claiming | SELECT, UPDATE on table |
Accepted design for 0.7; not shipped in the current release. This section records the privilege boundary the implementation and migration must satisfy.
The #401 implementation owns the narrow ADR-043 substrate required to make that boundary strict in
0.7: a pre-provisioned bounded execution owner, the exact complete_job manifest entry, transactional
ownership transfer and PUBLIC revocation, the application-role grant, and awa doctor validation.
The broader producer/executor/maintenance/admin capability split remains deferred under #452.
The generic ownership-transfer examples above describe today's broad-grant profile. Once the
strict capability profile ships, its generated ownership manifest must exclude complete_job and
other bounded-owner definers from blanket transfer to awa_owner; awa doctor treats such a
transfer as strict-profile drift.
Strict validation also audits the complete PostgreSQL role graph. It follows both transitive
inherited privileges and nested SET ROLE paths from every runtime, application, callback, and
admin login, and rejects any non-allowlisted path to the bounded execution owner, migrator, schema
owner, or a role that can reach them. Direct-membership checks alone are not sufficient. The
conformance matrix runs this audit on the oldest and newest supported PostgreSQL majors — a pair
that straddles PostgreSQL 16's separate membership SET/INHERIT/ADMIN semantics — so the newer
role model cannot be mistaken for the older one.
ADR-042 lets a handler commit application rows
and guarded Awa completion in one transaction. That transaction normally comes from a separate
application pool whose login can write the application's billing/inbox tables. Do not make that
login a member of awa_runtime: the current runtime role can directly mutate and TRUNCATE Awa
tables, which is far broader than caller-owned completion needs.
The completion migration installs complete_job as the canonical public finalization entry point.
Its exact argument signature,
FinalizationReceipt result, stale-token SQLSTATE, and schema-version compatibility semantics are
part of the ADR-036 SQL worker contract. Compatible implementation changes keep this name. Breaking
improvements require ADR-036 deprecation and ADR-041 expand/migrate/contract; a versioned
coexistence name may be used during migration without making version suffixes permanent policy.
complete_job is a hardened SECURITY DEFINER function owned by the bounded execution owner
defined by ADR-043. A queue-storage schema owner may be used only as a transitional, non-strict
fallback; it does not satisfy the hardened owner model, and awa doctor rejects that ownership
when the deployment declares the strict profile. Its migration:
- fixes
search_pathto trusted schemas withpg_templast and fully qualifies every referenced object; - accepts no caller-controlled schema, table, function, or SQL identifier;
- revokes
EXECUTEfromPUBLICin the same migration transaction that creates the function; - transfers the function to the pre-provisioned bounded execution owner before commit in a strict deployment, or explicitly records the schema-owner fallback as non-strict;
- performs only guarded completion of the supplied finalization token and never commits; and
- has the same hardened per-schema installation for a custom queue-storage schema.
The operator grants the application-worker login (or a dedicated NOLOGIN group role used by
those logins) only USAGE on the Awa/queue-storage schema and EXECUTE on the exact versioned
completion function. It receives no direct Awa table, sequence, maintenance-function, or
TRUNCATE privilege. The Rust complete_in_tx helper calls this same SQL function through the
application transaction, so Rust and non-Rust workers share one privilege and conformance boundary.
Provisioning grants by the exact complete_job(...) signature only after ownership transfer;
replacement reasserts the PUBLIC revocation and preserves or reapplies only that manifest-listed
ACL. Removal follows ADR-036 rather than silently redirecting the grant to an internal helper.
Function execution authorizes completion of any valid token presented by that trusted worker
application role. Do not grant it to producer-only roles, browser/admin clients, or public callback
ingress roles. awa doctor resolves the exact regprocedure signature and checks its owner,
SECURITY DEFINER flag, fixed search_path, PUBLIC revocation, and application-role ACL. Negative
tests prove the finalizer role cannot read or mutate Awa tables directly or redirect the function
through search_path, inherited membership, or a transitive SET ROLE chain.
For teams that manage PostgreSQL access declaratively, pgroles can maintain the role model as a YAML manifest:
profiles:
runtime:
grants:
- on: { type: schema }
privileges: [USAGE]
- on: { type: table, name: "*" }
privileges: [SELECT, INSERT, UPDATE, DELETE, TRUNCATE]
- on: { type: sequence, name: "*" }
privileges: [USAGE, SELECT]
- on: { type: function, name: "*" }
privileges: [EXECUTE]
default_privileges:
- on_type: table
privileges: [SELECT, INSERT, UPDATE, DELETE, TRUNCATE]
- on_type: sequence
privileges: [USAGE, SELECT]
- on_type: function
privileges: [EXECUTE]
schemas:
- name: awa
profiles: [runtime]
roles:
- name: awa_owner
login: false
- name: awa_migrator
login: true
password:
from_env: AWA_MIGRATOR_PASSWORD
- name: awa_runtime
login: true
password:
from_env: AWA_RUNTIME_PASSWORD
memberships:
- role: awa_owner
members:
- name: awa_migratorpgroles diff shows planned changes, pgroles apply converges. You can also run pgroles generate against an existing AWA database to produce an initial manifest.
If you're currently running everything as one superuser or app role:
- Create
awa_owner,awa_migrator,awa_runtimeas above - Transfer ownership to
awa_owner - Grant runtime privileges
- Update your migration tooling to use
awa_migrator - Update worker/serve connection strings to use
awa_runtime - Verify with
awa --database-url postgres://awa_runtime:... job list
This is additive — no schema changes, no downtime.
The current model requires broad table grants because compatibility triggers and helper functions
still use SECURITY INVOKER. ADR-042's completion function is a deliberately narrow
SECURITY DEFINER exception for an application role that must join business writes without gaining
runtime privileges; it does not reduce what the ordinary runtime needs.
ADR-043 defines the long-term tightening path. Awa
will not convert the entire function surface to SECURITY DEFINER: generic internal,
caller-controlled dynamic-SQL, arbitrary-schema, and DDL helpers would become excessive
privilege-escalation entry points. Instead, an allowlisted set of producer, executor, maintenance,
admin, callback, and finalizer capability functions runs under a bounded NOLOGIN execution owner.
Maintenance partition reclamation is the sole relation-dispatch exception: it accepts only a
bounded ring slot, verifies each manifest-listed Awa child by catalog OID/namespace/owner/attachment,
uses a short lock timeout, revalidates every target's catalog identity again after the locks are
held, and may issue only LOCK TABLE ONLY ... ACCESS EXCLUSIVE plus
TRUNCATE TABLE ONLY ... CONTINUE IDENTITY RESTRICT after the reclaimability recheck. It never accepts a caller relation or SQL fragment. Internal helpers remain
inaccessible, PUBLIC EXECUTE is revoked, and broad table grants are removed only after an ADR-041
capability rollout. This work is tracked in #452.
Awa is one process binary, but for production it splits into several deployable roles. Each role has a different exposure profile, and mixing them onto the same listener is the most common source of operational risk. See ADR-027 for the design rationale and docs/http-callbacks.md for the per-role deployment shape.
| Role | Purpose | Exposure | Mutates job state? |
|---|---|---|---|
Admin UI / API (awa serve) |
operator inspection and mutation — jobs, queues, runtime, DLQ, stats | private operator network | yes |
Callback receiver (awa callbacks serve or user-owned router) |
complete / fail / heartbeat for HttpWorker async mode and external systems |
public or partner-facing, signed | yes |
Workers / dispatchers (awa::Client with registered workers) |
claim jobs, execute handlers, dispatch via HttpWorker |
internal | yes |
| Maintenance runtime (background tasks on any worker process; future: dedicated role per ADR-028) | promotion, rescue, pruning, metadata refresh | internal | yes |
| Database | storage and coordination | private | yes |
A single development setup can collapse these onto one process: awa serve runs admin + callback receiver, an embedded awa::Client runs workers, and Postgres is reachable on localhost. Production deployments should split at least the admin UI from the callback receiver — the admin surface stays on the operator network, the receiver lives wherever it needs to be reachable from the function (often public).
- All-in-one dev:
awa serve+ an embeddedawa::Client. Admin UI and callback receiver share the same listener with permissive CORS. Fine for local development; never run this on a public listener. - Private admin + public callback receiver:
awa serveon a private VPC subnet,awa callbacks serve --callback-hmac-secret …on an external load balancer. The receiver router omits static UI assets, the admin REST routes, and permissive CORS. - User-owned callback API: mount the three callback ingress routes inside your existing FastAPI / axum / Flask app, using
awa_model::callback_contract::verify(Rust) orawa.callback_contract.verify(Python) so the signature contract cannot drift. Seedocs/callback-receivers.md. - Receiver + maintenance-only runtime: the receiver handles ingress while a dedicated runtime instance runs promotion / rescue / pruning. Workers stay on their own deployment. Maintenance-only runtime is tracked in ADR-028.
- HTTP-worker deployments: the worker process still runs an
awa::Client— it claims jobs, calls the function, and registers the callback. The receiver is a separate listener. A function endpoint without a corresponding dispatcher process will never see jobs.
awa serve is an operator surface: it bundles the admin REST API, the React dashboard, the static fallback, permissive CORS, and (today) the callback receiver routes behind a single router. Treat it like a database admin console:
- Put it behind your normal authentication and authorization layer.
- Restrict network access with ingress policy, firewall rules, or private networking.
- Prefer binding to localhost or an internal service address unless you explicitly need external access.
When callbacks must be externally reachable but the admin surface must stay private, run them on separate listeners using awa callbacks serve or a user-owned receiver — the admin endpoints simply do not exist on those routers.
The callback receiver exposes three mutating ingress endpoints, regardless of which deployable role hosts them:
POST {prefix}/:callback_id/completePOST {prefix}/:callback_id/failPOST {prefix}/:callback_id/heartbeat
{prefix} defaults to /api/callbacks, matching the historical awa serve shape so existing deployments keep working unchanged. It is configurable on both sides:
- Worker side:
HttpWorkerConfig::callback_path_prefix awa callbacks serveside:--path-prefix/AWA_CALLBACK_PATH_PREFIX- Custom-receiver side: mount your routes at whatever prefix you want and pass that prefix back to the worker config
These endpoints mutate job state and must not be exposed without protection. The full HTTP worker flow, callback payloads, and signature contract are documented in HTTP workers and callback signatures.
Awa supports per-callback request authentication with a 32-byte BLAKE3 keyed hash.
- Configure the callback receiver with
--callback-hmac-secret <64-hex-chars>orAWA_CALLBACK_HMAC_SECRET. - Configure
HttpWorkerConfig.hmac_secretwith the same 32-byte key. - The worker signs the callback ID and sends the signature as
X-Awa-Signature. - The function normally forwards that same header when it calls Awa back.
- The callback receiver verifies that header before accepting
complete,fail, orheartbeat.
Example:
export AWA_CALLBACK_HMAC_SECRET=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
awa --database-url "$DATABASE_URL" serve --host 0.0.0.0 --port 3000If no callback secret is configured, signature verification is disabled. That is acceptable only for trusted internal deployments where the callback receiver is already protected by network boundaries or an authenticating proxy.
The option name says hmac for operational familiarity, but the implementation uses BLAKE3 keyed hashing over the callback ID string, not RFC HMAC.
When you host the callback ingress routes inside your own application (FastAPI, axum, Flask, etc.), reuse the shared helpers in awa::callback_contract (Rust) or awa.callback_contract (Python) rather than re-implementing the signature algorithm. The Python wrappers are thin PyO3 bindings around the same Rust functions, with a pinned BLAKE3 test vector asserted from both sides so the bindings cannot drift.
See callback receivers for end-to-end custom-axum and FastAPI examples.
- Rotate callback secrets like any other shared secret.
- Use different secrets per environment.
- Prefer HTTPS/TLS termination in front of any externally reachable callback receiver.
- Avoid logging callback signatures or other shared-secret material.