Every content mutation in StaticOwl is hash-chained at the engine. The chain is verifiable on demand by anyone with read access to the site — auditors, compliance teams, the customer themselves. This document is how.
The contract: if verify-audit-chain returns status: "ok", the engine asserts no record was inserted, modified, or removed since the chain began without the change being visible. If it returns anything else, the body explains what was found.
Two ways to verify
1. The CLI binary (recommended for operators + auditors)
npx verify-audit-chain --site site:acme --key so_…
# or
node bin/verify-audit-chain.mjs --site site:acme --key so_…
Runs against the CMS (default endpoint: https://cms.staticowl.com). The CMS validates the so_ key, confirms you have access to the site's underlying graph, then forwards to the engine with its own engine-direct credential. You never need a gq_ key.
Flags
| Flag | Default | Meaning |
|---|---|---|
--site <id> |
(required) | Site id, e.g. site:acme. |
--key <key> |
$STATICOWL_API_KEY |
A site-scoped CMS key (so_…). |
--endpoint <url> |
https://cms.staticowl.com |
CMS base URL. Point at staging or a self-hosted CMS if needed. |
--checklist |
off | Also run db.compliance.checklist() and print the per-feature compliance posture (WORM, audit chain, retention, …). |
--json |
off | Emit machine-readable JSON to stdout instead of the human report. |
Exit codes
| Code | Meaning |
|---|---|
0 |
Chain verified clean. |
1 |
Chain is broken — body has the reason. Treat as an incident. |
2 |
Auth / access failure (bad key, no access to the site). |
3 |
Engine unreachable or returned a malformed payload. |
2. The CMS REST endpoints (when scripting)
The binary is a thin wrapper around two endpoints. Call them directly when integrating into your own audit pipeline.
GET /api/audit/verify?graph=<name>
Returns the engine's AuditStatus payload unchanged:
{
"graph": "staticowl_8f5a2b3c…",
"status": "ok",
"records": 142_088,
"last_seq": 142_088,
"last_hash": "9a8b…",
"verified_at": "2026-06-14T12:00:00.000Z",
"reason": null
}
Fields:
| Field | Meaning |
|---|---|
status |
"ok" (chain walks cleanly), "broken" (a record's stored hash disagrees with the recomputation), or "empty" (no records yet). |
records |
Total records in the chain. |
last_seq |
Sequence number of the most recent record. |
last_hash |
Hash of the most recent record — pin this in your audit log to detect future tampering. |
verified_at |
When the engine completed the walk. |
reason |
On "broken", a one-line description (e.g. "record 4283: stored hash mismatch"). null otherwise. |
HTTP status mirrors the verdict: 200 whether the chain walks cleanly or is broken (the status field carries the verdict; both are successful verifications), 4xx for auth/access problems, 5xx for engine failures.
POST /api/audit/checklist?graph=<name>
Returns the result of CALL db.compliance.checklist() — one row per compliance feature:
{
"rows": [
{ "feature": "audit_chain", "status": "active", "version": "v1",
"last_verified_at": "2026-06-14T12:00:00Z", "message": null },
{ "feature": "worm_mode", "status": "active", "version": "v1",
"last_verified_at": null, "message": null },
{ "feature": "retention_policy", "status": "inactive", "version": null,
"last_verified_at": null, "message": "no retention policy configured" }
]
}
Use this to assert a known compliance posture in CI or in a regulator-facing scheduled job.
Auth + tenant isolation
The proxy validates that the so_ key's owner has HAS_ACCESS to a Site whose graphName matches the requested graph. A leaked so_ key for tenant A cannot verify graphs owned by tenant B; the response is 404 graph not found (same shape whether the graph doesn't exist or is in another tenant — we don't leak the difference).
Engine-direct gq_ keys still work against the engine's own /graphs/:graph/audit/verify if you have one, but most customers should never see one. The CMS proxy is the canonical path.
What's actually being verified
The engine writes one record per content mutation: the operation, the after-state, the prior record's hash, and the new hash (hash = sha256(prevHash || serializedOp)). verify-audit-chain walks the chain from genesis, recomputes each hash, and stops at the first mismatch.
A chain in "ok" state means:
- Every record's stored hash matches the recomputation.
- The chain has no gaps (every
seqis present). - The most recent record's hash is
last_hash.
A chain in "broken" state means one of:
- A record's payload was modified after insertion (silent rewrite).
- A record was deleted from the underlying store (tombstone, manual cleanup, storage corruption).
- A record was inserted out of band without the engine recomputing the chain (engine bug, custom write path).
In production, "broken" is always a tooling/operator/incident concern — not something that resolves on its own. Treat it like a failed build gate.
Operational guidance
Pin last_hash somewhere durable
The protective property of the chain is "tampering becomes visible." That requires someone — you, an auditor, an external attestation service — to know what the hash was yesterday so a discrepancy today is detectable. Otherwise an attacker who rewrites the chain consistently from genesis is invisible.
In practice:
- Pin yesterday's
last_seq+last_hashin your SIEM, ticketing system, or a write-once audit log. - For regulated workloads, the CMS supports WORM mode (
db.compliance.checklist()will report it). See the parent project's audit-chain ADR if you need to dig in.
Run on a schedule
For a regulated customer, the typical cadence is:
- Daily: scheduled CI job runs the binary, alerts on non-zero exit.
- Pre-deploy: CMS pre-deploy gate calls the endpoint; deploy aborts if the chain is broken.
- Quarterly: full chain walk + checklist captured into the compliance artifact bundle.
Don't conflate compliance status with chain status
/api/audit/verify is "is the chain walkable and untampered?" — that's an integrity property.
/api/audit/checklist is "what compliance posture is the site configured for?" — that's a configuration property.
A site can have a verified chain but no WORM, or WORM enabled with a healthy chain, etc. Treat them as orthogonal answers.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Binary exits with 2 (auth) |
so_ key wrong, expired, or doesn't have site access |
Verify the key works against /api/sites first. |
Binary exits with 3 (unreachable) |
CMS endpoint wrong; CMS itself down | curl https://cms.staticowl.com/api/health. |
status: "broken" |
Real chain damage | Open an incident. Engine logs at the seq named in reason. Do not silently re-stamp; that defeats the purpose of the chain. |
Endpoint returns 404 graph not found |
so_ key isn't authorized for that site |
Confirm the site's graphName and the user's HAS_ACCESS edges. |
See also
- HTTP API →
/api/audit/*— same endpoints reference shape - Features → Audit + Replay — the customer-facing pitch
packages/server/src/routes/audit.ts— the CMS proxy implementationbin/verify-audit-chain.mjs— the binary source- Parent project's storage docs — the engine-side hash-chain mechanics