metastar/meta-core
4
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 1 Wiki Activity

v2.0.0: claims model - signed MMUR2 votes in a truth DB, merged view in a rebuildable cache DB; POST /api/claims, claims and merge routes, unsigned writes answer 410
Build and Push Docker Image / build (push) Successful in 5m1s
Details

Browse Source
This commit is contained in:
Worph
2026-10-25 17:36:00 +00:00
parent b0adefb97b
commit 84af298792
78 changed files with 7203 additions and 5617 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/meta-core-architecture.md
+315 -103
View File
@@ -1,15 +1,22 @@
# Meta-Core Architecture
> **meta-core v2.0.0 — the claims model.** Implementation basis:
> [`docs/implementation/metacore-claims-model.md`](../../../docs/implementation/metacore-claims-model.md)
> (meta-root). Where this file and that one disagree, that one wins.
## Document Purpose
This document describes the architectural design of **meta-core**, the
standalone container that owns the metadata store (an embedded SQLite
file; Redis until 2026-10), the leader-info file, the HTTP+SSE
standalone container that owns the metadata store (two embedded SQLite
databases since v2.0.0 — *truth* and *cache*; a single SQLite file in
v1.x; Redis until 2026-10), the leader-info file, the HTTP+SSE
metadata surface, and a WebDAV file server. It covers the leader gate,
storage data model, API design, the SSE event surface, and the integration
contract with other MetaMesh services.
the claims data model and its write/apply pipeline, API design, the SSE
event surface, and the integration contract with other MetaMesh services.
For the on-disk key layout, see
For the engine design (SQLite, group commit, measurements), see
[metacore-storage-engine.md](../../../docs/project-architecture/metacore-storage-engine.md);
for the on-disk layout (tables of both databases), see
[metadata-storage-structure.md](metadata-storage-structure.md). For the
external-access design (store lockdown, SSE wire contract, auth rules),
see [api-mediated-access.md](api-mediated-access.md). For the UUID-rooted
@@ -24,7 +31,7 @@ view of file metadata:
| Service | Language | Role |
|---|---|---|
| **meta-sort** | TypeScript | Watches folders, runs plugins, writes metadata via meta-core's HTTP API. |
| **meta-sort** | TypeScript | Watches folders, runs plugins; each plugin signs its own claims and posts them to meta-core. |
| **meta-fuse** | TypeScript / Rust | Serves the virtual filesystem from metadata. |
| **meta-stremio** | Python | Stremio addon for HLS streaming. |
| **meta-dup** | TypeScript | Duplicate detection. |
@@ -32,8 +39,14 @@ view of file metadata:
meta-core sits below all of these as the single owner of:
1. **The metadata store** — an embedded SQLite file
(`/meta-core/db/meta-core.db`) holding metadata and the event streams.
1. **The metadata store** — two embedded SQLite databases: **truth**
(`/meta-core/db/meta-core.db`, backed up) holding the signed claims, and
**cache** (`/meta-core/db-cache/cache.db`, never backed up) holding every
index and derived row — a v1-layout `kv` table (the merged view as
`file:<rid>/<field>` rows, lookup, counters, event streams, service
scratch KV) with its field index and FTS, plus the `claim_by_pid` and
`local_file` tables. (v2.0.0 is implemented, uncommitted and not yet
deployed as of 2026-10-25.)
2. **The leader-info file** — the one place services look to discover meta-core's URLs.
3. **The HTTP+SSE surface** — the only way external services touch the data plane.
4. **A WebDAV server** — file access for plugins and other services.
@@ -44,7 +57,12 @@ meta-core sits below all of these as the single owner of:
> through meta-core's HTTP+SSE surface.
This is the contract that PR D of [api-mediated-access.md](api-mediated-access.md)
landed and that the rest of this document assumes.
landed and that the rest of this document assumes. v2.0.0 adds a second:
> Every metadata value is a **signed vote** — `claim(uid, cid, key) → value`
> + `ts` + `sig`. The only write is `POST /api/claims`. meta-core verifies
> every claim and **never signs one**: it holds no writer key, and the
> signature is the write authentication.
---
@@ -68,13 +86,15 @@ database file and the writable state under `/meta-core`.
│ ├─ rclone rcd (mount daemon) │
│ ├─ mount-watcher.sh (rclone mount lifecycle) │
│ └─ meta-core (Go) ──► HTTP + SSE + WebDAV on :9000 │
│ + embedded SQLite (db/meta-core.db) │
│ + embedded SQLite: truth (db/meta-core.db) │
│ and cache (cache/cache.db) │
└────────────────────────────────────────────────────────────────────────┘
│
┌────────────────────────────┴────────────────────────────┐
│ /meta-core (shared volume) │
│ locks/kv-leader.{lock,info} │
│ db/meta-core.db (+ -wal, -shm) │
│ db/meta-core.db (+ -wal, -shm) truth, backed up │
│ cache/cache.db (+ -wal, -shm) cache, never backed │
│ services/*.json │
│ mounts/{mounts.json, errors/} │
│ watchers.json │
@@ -203,67 +223,166 @@ Other services read this file, then call `GET /urls` for full discovery
## Schema-Version Gate
Before serving any traffic, `cmd/meta-core/main.go:54-61` calls
`storage.EnsureSchemaVersion`. The sentinel lives in the store at
`meta-core:schema-version` (`internal/storage/schema_version.go`). The
gate refuses to boot when the existing store layout predates the
UUID-rooted schema, so legacy `file:midhash256:*` data can't leak into
the current code paths. In alpha there is no automated migration —
operators wipe with `pnpm run clean:all` and restart.
Before serving any traffic, `cmd/meta-core/main.go` opens both databases
and calls `storage.EnsureSchemaVersion` (before the cache catch-up). In
v2.0.0 the sentinel is the `schema-version = "3"` row of the truth
database's `meta` table. The gate:
- **explicitly refuses a v1.x store** — a truth file holding the v1 `kv`
table (`meta-core:schema-version = "2"`), which the default path still
points at — with a message naming `tools/claims-migrate`;
- refuses a store with claims but no sentinel, or any other version;
- stamps a fresh store with `"3"`.
So no pre-claims data leaks into the current code paths. The meta-core
**service binary contains no migration code**: watch.nsl.sh is converted
once by the separate one-off tool `packages/meta-core/tools/claims-migrate`
(not built into the image; implementation basis §11); every other box and
dev stack starts empty (operators wipe with `pnpm run clean:all` and
restart).
---
## Data Model
### Two concepts: metadata vs. file bytes
### The one concept: signed votes
```
┌────────────────────────────────────────────────────────────────────┐
│ METADATA (embedded SQLite, kv table) │
│ │
│ Root: opaque UUIDv7 (Crockford Base32; ULID layout, 26 chars) │
│ Storage: flat string keys │
│ file:<uuid>/<property> → string value │
│ file:<uuid>/cids/<bareCid> → "true" (key-set) │
│ dup:<uuid>/<path> → "" (one per path) │
│ │
│ No stored canonical_cid — derived by rank on read. │
│ │
│ Aliases (reverse index, one key each): │
│ cid:<bareCid> → <uuid> │
│ │
│ Index: idx:roots/<uuid> → "" (one per root) │
│ │
│ Mintable without a local file (remote / deleted entries OK). │
│ Resolvable by any registered CID in O(1). │
└────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────┐
│ FILE BYTES │
│ │
│ Location: /files volume (shared across containers). │
│ Access: path resolution via metadata, served via /webdav/*, │
│ /file/{cid} (reverse-lookup), or /data/{hash}. │
│ Path stored as filePath in the metadata, relative to /files. │
└────────────────────────────────────────────────────────────────────┘
claim(uid, cid, key) → value + ts + sig
```
### Why UUID-rooted?
- `uid` — the signer (a secp256k1 public key, `z…` base58btc). One key per
writing container (each meta-sort plugin, each self-publishing feeder,
gateway core, meta-share, meta-search) and one per user.
- `cid` — the subject, a content CID; never a rid.
- `key` — the field (`METADATA_KEYS.md`); key-set members live inside the
key (`genres/Drama`, `cids/<cid>`).
- `value` — text (public), bytes (signed + encrypted envelope) or null
(retraction, kept as a tombstone).
- `ts` — milliseconds; it **is** the version and only increases per
`(uid, cid, key)`.
- `sig` — MMUR2: secp256k1 ECDSA (compact 64 bytes) over SHA-256 of
`"MMUR2\n" ‖ canonical CBOR [cid, uid, key, value, ts]`.
Roots were previously content-hash tokens (`file:midhash256:abc/...`),
which privileged one hash algorithm. UUID-rooted storage treats every CID
symmetrically as an alias, makes `GET /api/meta/<cid>` O(1) for any
registered CID, and lets new digest types be added without rewriting root
keys. See [uuid-rooted-metadata.md](uuid-rooted-metadata.md) for the
design rationale and [metadata-storage-structure.md](metadata-storage-structure.md)
for the on-disk layout.
The old shared record (`file:<rid>/<field>`) and the old User Data Layer
store (`udl:rec:*`) both collapse into this shape. The store distinguishes
only plaintext (signed) from ciphertext (signed + encrypted); "personal vs
shared" is app logic.
### Two databases
```
┌──────────────────────────────────────────────────────────────────────┐
│ TRUTH /meta-core/db/meta-core.db (backed up; PKs only) │
│ │
│ principal(pid, uid) uid ↔ local surrogate pid │
│ claim(cid, key, pid → value, ts, sig, seq) │
│ the votes; PK (cid, key, pid) │
│ cid_group(cid → rid) hidden local record identity │
│ meta(name → value) schema-version, last_seq │
└──────────────────────────────────────────────────────────────────────┘
│ apply loop (seq order)
▼
┌──────────────────────────────────────────────────────────────────────┐
│ CACHE /meta-core/db-cache/cache.db (never backed up; rebuildable)│
│ │
│ kv(k → v) the v1 layout, every row derived: │
│ file:<rid>/<field> the merged view every read uses │
│ cid:<cid> cid → rid lookup │
│ idx:roots/ idx:fts/ idx:count/ idx:schema/ roots, FTS rowid, │
│ counters (search totals, stats) │
│ ev:<stream>/…, evhwm:… /api/events/meta + /files backlog │
│ meta:applied_seq, meta:rebuilt_at apply-loop bookkeeping │
│ anything else /api/kv/value scratch KV │
│ kv_field_v (index), fts filter + free-text search │
│ claim_by_pid(pid, cid, key) "everything uid X wrote" (?as=, │
│ UDL facade, identity retraction) │
│ local_file(path → midhash, size, mtime) watcher facts, unsigned │
└──────────────────────────────────────────────────────────────────────┘
```
Disaster recovery is "delete the cache, replay truth". `META_CORE_DB_PATH`
names the truth file, `META_CORE_CACHE_PATH` the cache file. The cache keeps
the v1 `kv` layout so every v1 reader (record reads, field index, FTS,
counters, SSE + snapshot) runs unchanged on derived rows; `nsrc` is computed
by the merge (`POST /api/merge`), not stored. Full schema: implementation
basis §5.
### Records: cids, groups and the hidden rid
A vote never mentions a rid. A **rid** (UUIDv7, 26-char Crockford, as in
v1.x) is a local name for a group of cids known to be the same content,
held in `cid_group`:
- A new cid with no link gets its own group and a fresh rid.
- A `cids/<other>` vote on cid X links X's group and `<other>`'s group; the
group with the older rid survives, the losing rid gets a `del` event per
key and the survivor `set` events (no alias table). Retracting the link
does **not** split the group (phase 1); the record view synthesises
`cids/<cid> = "true"` for every cid of the group from `cid_group`.
- Reading record R = every vote on every cid of R's group, merged per key;
within a group one signer counts once per key.
- rids are never signed and never travel; consumers that hold them
(meta-sort, meta-fuse, meta-stremio) keep working because `cid_group`
lives in truth.
### Merge
Per `(rid, key)`, over the group's votes, each signer's newest vote counted
once (`internal/storage/merge.go`, shared by the apply loop, `?as=` and
`POST /api/merge`):
1. the viewer's own live text vote (`?as=<uid>`, applied at read time,
never stored);
2. retractions and ciphertext drop out — only text merges;
3. most distinct signers for the normalised value (**trim + case-fold
only**, no NFC) — `nsrc` is that count (computed, not stored);
4. tie → oldest `ts`;
5. tie → byte-wise smallest value.
The stored value is the winning vote's exact text. Key-sets need no special
code (each member is a key, present while one vote supports it). A
different kind of fact gets a different key, so a guesser (e.g.
filename-parser) never competes with a lookup on the same key.
### Box-local file facts
`filePath`, size, `mtimeNano` and midhash are **not claims** — meta-core
cannot sign, and the disk can always be rescanned. The file watcher and
`POST /api/files/register` write them into the cache's `local_file` table
(and the midhash into `cid_group`); they are joined into the record view
unsigned and never reach the claims API or the swarm. When several local
paths hold the same content, `filePath` is the lexicographically first one
(size and mtime from the same row); a signed `sizeByte` claim wins over the
on-disk size.
### File bytes
```
Location: /files volume (shared across containers).
Access: path resolution via local_file, served via /webdav/*,
/file/{cid} (reverse-lookup), or /data/{hash}.
Path: filePath (from local_file) in the record view, relative to /files.
```
### History: UUID-rooted flat keys (v1.x)
Until v2.0.0 a record was a set of flat keys under an opaque UUID root —
`file:<uuid>/<property>`, `file:<uuid>/cids/<bareCid>` key-sets,
`dup:<uuid>/<path>`, a reverse index `cid:<bareCid> → <uuid>` and a root
index `idx:roots/<uuid>`, all in one kv table. Roots had previously been
content-hash tokens (`file:midhash256:abc/...`); UUID roots treated every
CID symmetrically as an alias. The rid of v2.0.0 is that UUID, kept hidden
and local; migrated records kept their UUIDs. See
[uuid-rooted-metadata.md](uuid-rooted-metadata.md) (the *why*) and the
History note of [metadata-storage-structure.md](metadata-storage-structure.md).
### Path resolution
`filePath` values stored in metadata are absolute paths under the `/files`
volume (e.g. `/files/watch/movies/Movie.2024.mkv`). The `/files` prefix is
itself a per-host detail — moving the volume only requires re-mounting it
at the same path inside the container.
`filePath` values are absolute paths under the `/files` volume (e.g.
`/files/watch/movies/Movie.2024.mkv`). The `/files` prefix is itself a
per-host detail — moving the volume only requires re-mounting it at the
same path inside the container.
---
@@ -282,16 +401,16 @@ Bootstrap / discovery
GET /neighbors (alias: /api/neighbors)
GET /api/stats
Metadata (primary surface for other services)
Claims (the only metadata write path)
POST /api/claims signed claim batch → per-claim status
POST /api/merge {claims, as?} → {fields, nsrc, rejected}; no write
GET /api/meta/{cid}/claims {rid, cids, claims} of the cid's group (?key=, ?uid=; 404 unknown)
GET /api/metadata/{rid}/claims same, addressed by rid
Metadata reads (merged view + local_file; optional ?as=<uid>)
GET /meta
GET /meta/{hash}
PUT /meta/{hash}
PATCH /meta/{hash}
DELETE /meta/{hash}
GET /meta/{hash}/{key...}
PUT /meta/{hash}/{key...}
DELETE /meta/{hash}/{key...}
POST /meta/{hash}/_add/{key...}
CID-addressed (public, auth-bypassed at the perimeter)
GET /api/meta/{cid}
@@ -301,24 +420,45 @@ CID-addressed (public, auth-bypassed at the perimeter)
HEAD /data/{hash}
GET /data/{hash}/path
Editor / KV / schema / snapshot
Editor / KV / schema
GET /api/metadata/hash-ids
GET /api/files/tuples
POST /api/files/register (fills local_file + cid_group)
GET /api/metadata/list
POST /api/metadata/search
POST /api/metadata/batch
POST /api/metadata/clear
{GET,PUT,DELETE} /api/metadata/{hashId}
{GET,PUT} /api/metadata/{hashId}/property
GET /api/kv/{info,keys,tree,search,find,value}
PUT /api/kv/value
DELETE /api/kv/value
GET /api/kv/key/{key...}
POST /api/metadata/clear (wipes both databases)
GET /api/metadata/{hashId} (hashId = rid)
GET /api/metadata/{hashId}/property
GET /api/kv/{info,keys,tree,search,find} (read-only browser over the cache kv table)
{GET,PUT,DELETE} /api/kv/value (service scratch keys only; reserved prefixes refused)
GET /api/kv/key/{key...} (read-only)
GET /api/schema
POST /api/schema/rescan
GET /api/snapshot/export
POST /api/snapshot/import
POST /api/snapshot/wipe
User Data Layer (facade over claims; records are MMUR2)
{GET,PUT} /api/udl/record
PUT /api/udl/records
GET /api/udl/user/{uid}/{key/{key},cid/{cid},all,count}
GET /api/udl/cid/{cid}/{users,aggregate}
GET /api/udl/users/stats (every signer with live claims, plugin keys included)
Account-key identity service (kept in v2.0.0; removal is a separate ticket)
{GET,DELETE} /api/identity (DELETE signs + stores a retraction of every live claim first)
GET /api/identity/{accounts,aead-key}
POST /api/identity/{challenge,generate,import,reveal,sign,sign-batch}
(sign/sign-batch `format`: compact | der; default
compact for "MMUR2\n" payloads, DER otherwise)
Removed in v2.0.0 — answer 410 Gone, pointing to POST /api/claims
PUT/PATCH/DELETE /meta/{hash}
PUT/DELETE /meta/{hash}/{key...}
POST /meta/{hash}/_add/{key...}
PUT/DELETE /api/metadata/{hashId}
PUT /api/metadata/{hashId}/property
POST /api/metadata/batch
Removed in v2.0.0 — no snapshot format
GET /api/snapshot/export, POST /api/snapshot/{import,wipe}
SSE event streams (HTTP-mirror of the store's event streams)
GET /api/events/files
@@ -366,6 +506,13 @@ peers and unauthenticated tooling can reach them):
- (`/api/events/files`, `/api/events/meta` are also auth-bypassed but
**should not** be exposed publicly via Caddy; they're inside-only.)
Since v2.0.0 `/meta/*` serves reads only — its write verbs answer `410
Gone` — so the bypass no longer exposes an unsigned write. Writes go to
`POST /api/claims`, where the claim signature (not the perimeter) is the
write authentication. `POST /api/claims`, `POST /api/merge` and
`GET /api/meta/{cid}/claims` are container-to-container routes, not routed
through Caddy, like the v1.x write routes were.
See `docs/api-mediated-access.md` "Auth" for the rationale.
### SSE event streams — wire contract
@@ -389,9 +536,46 @@ the kv table (`internal/kv/stream.go`). The contract
`event: gap` payload before resuming from the oldest available entry.
External services use SSE with `Last-Event-ID` as the resume primitive.
`meta:events` entries are appended by the storage engine in the same
transaction as the change they describe; the Redis-era keyspace-notification
hop through `MetaPublisher` is gone.
Since v2.0.0 `meta:events` describes changes of the **merged** view, same
`{type: set|del, key, ts}` shape: the cache engine appends an
`ev:meta:events/…` row for every change of a merged `file:` row, in the same
cache transaction as the change (both streams are v1 stream rows of the
cache `kv` table). A cache rebuild
emits no per-key events — consumers get `event: gap` and take a snapshot;
event IDs stay time-based and keep rising. *History:* v1.x appended them in
the same transaction as the `file:` key change; the Redis-era
keyspace-notification hop through `MetaPublisher` is long gone.
### Write path and apply loop
```
POST /api/claims ──► per claim, before the writer:
0. validate (required fields, reserved keys, ts range) → else bad_request
1. same ts + sig as stored? → unchanged (no verify)
2. same value as stored (any ts)? → unchanged; stored row + older ts kept
3. stored ts ≥ incoming ts? → stale
4. verify MMUR2 signature → else bad_signature
──► accepted claims, one group-committed truth transaction:
5. re-check 1–3 under the writer
6. uid → pid (principal; insert on first sight)
7. upsert claim (tombstone if null), seq = ++last_seq
8. cid_group: mint rid for unknown cid; text cids/* links groups
(a retraction never splits)
9. commit (last_seq in truth meta); answer once the cache applied it
──► apply loop, in seq order, into the cache:
sync cid:/cids/<cid>/idx:roots rows → claim_by_pid
→ re-merge (rid, key) from truth PK range reads
→ set/delete file:<rid>/<key> (+ local_file for filePath/sizeByte/mtimeNano)
⇒ meta:events row + counter delta in the same tx
→ FTS row on a haystack change → meta:applied_seq
```
The verifier lives in the write path (secp256k1 from `internal/identity`);
meta-core holds **no** writer key and never signs a claim. Response: one
status per claim — `accepted` · `unchanged` · `stale` · `bad_signature` ·
`bad_request`. Request/response shapes and the 410 contract for removed
routes: [api-mediated-access.md](api-mediated-access.md) "Write contract —
signed claims".
### WebDAV URLs
@@ -421,7 +605,8 @@ WebDAV caching is handled by nginx's `proxy_cache` (see
| Single binary | Static compilation |
| Low resource | ~10-20 MB resident |
| HTTP server | stdlib `net/http` + gorilla/mux |
| Metadata store | embedded SQLite via `modernc.org/sqlite` (pure Go, CGO off) |
| Metadata store | embedded SQLite via `modernc.org/sqlite` (pure Go, CGO off) — truth + cache databases |
| Claim verification | secp256k1 ECDSA (`internal/identity`) |
| WebDAV | `golang.org/x/net/webdav` |
### Repo layout
@@ -434,12 +619,12 @@ packages/meta-core/
│ ├── cid/ CID parsing + algorithm ranking
│ ├── config/ env-driven configuration
│ ├── discovery/ service registry + dead-service cleaner
│ ├── kv/ SQLite kv engine + event streams
│ ├── identity/ secp256k1 (claim verification; account-key service)
│ ├── kv/ SQLite engine (single writer, group commit) + event streams
│ ├── leader/ LeaderInfoProvider (no election logic)
│ ├── mounts/ rclone mount manager + handlers + stats
│ ├── schema/ per-field schema from transactional counters
│ ├── snapshot/ snapshot export / import / wipe
│ ├── storage/ flat-key records + search + UUID + CID reverse index + schema gate
│ ├── storage/ claims write path, apply loop, merge, cid_group, search, schema gate
│ ├── watcher/ scanner + dispatcher + midhash + state
│ ├── watchers/ polling-based watcher configs
│ └── webdav/ /webdav/* handler
@@ -460,15 +645,22 @@ packages/meta-core/
```
1. Load configuration from environment
2. Build LeaderInfoProvider (no election; bash gate already won)
3. Open the SQLite store (META_CORE_DB_PATH or /meta-core/db/meta-core.db)
4. EnsureSchemaVersion (abort if the store layout is stale)
5. Start service discovery + dead-service cleaner
6. NewServer + Server.Start (boots watcher, mounts, WebDAV, API)
7. EnsureCounters (one-time build of the derived counters
on a store that predates them)
8. Rebuild FTS rows in the background if the FTS table is empty
3. Create the cache folder; open the truth database (META_CORE_DB_PATH or
/meta-core/db/meta-core.db) and the cache (META_CORE_CACHE_PATH or
/meta-core/db-cache/cache.db) (storage.Connect)
4. EnsureSchemaVersion (abort on a v1 kv store or an unknown layout)
5. storage.Start — cache catch-up (meta:applied_seq ≥ last_seq → nothing; behind →
replay claims with seq > applied_seq, or a full
rebuild if a cids/* link is among them; no
cache → full rebuild), then the apply loop
6. Derived counters ready
7. Start service discovery + dead-service cleaner
8. NewServer + Server.Start (boots watcher, mounts, WebDAV, API)
9. Wait for SIGINT/SIGTERM
10. Shutdown in reverse order
(v1.x ran `EnsureCounters` and a background FTS rebuild here; in v2.0.0
counters and FTS are cache rows maintained with every merged-row change.)
```
### Process management
@@ -489,10 +681,25 @@ internal crashes don't trigger re-election.
### Store durability
The SQLite file runs in WAL mode with `synchronous=NORMAL`: every
committed transaction survives a meta-core process crash. A record write — fields, root index, CID aliases,
FTS row and its events — is one transaction, so a crash never leaves a
half-written record.
Both SQLite files run in WAL mode; truth with `synchronous=NORMAL` (every
committed transaction survives a meta-core process crash), the cache with
`synchronous=OFF` (it is rebuildable). A claim batch
commits to truth in one transaction; the apply loop then commits the
derived cache changes. **A crash between the two commits leaves the cache
behind, never wrong**: at boot, if the cache's `applied_seq` is below
truth's `last_seq`, meta-core scans truth's `claim` table for
`seq > applied_seq` (a full scan — there is no seq index; it only happens
after an unclean shutdown) and applies them — or rebuilds the cache in full
if one of them is a `cids/*` link.
### Cache loss
The cache is never backed up and is fully derived (except `local_file`,
which a rescan of `/files` refills, and the service scratch keys, whose
loss costs a refetch). A missing cache folder (no `meta:applied_seq`) means
meta-core rebuilds every derived row from truth and stamps
`meta:rebuilt_at`. A rebuild emits no per-key events;
SSE consumers get `event: gap` and re-snapshot.
### Container crash
@@ -503,8 +710,9 @@ the container."
### Schema mismatch
`EnsureSchemaVersion` refuses to boot. Operators wipe with
`pnpm run clean:all` (or equivalent) and restart. This is an alpha-stage
hard-fail; there is no automated migration path.
`pnpm run clean:all` (or equivalent) and restart, or — once, on
watch.nsl.sh — convert a v1.x store with `tools/claims-migrate`. This is an
alpha-stage hard-fail; the service never migrates automatically.
### Shared volume unavailable
@@ -525,7 +733,8 @@ via `/health`; metadata I/O and SSE event streams stop.
| `SERVICE_VERSION` | `1.0.0` | Reported in `/status`. |
| `API_PORT` | `8180` | External port baked into `baseUrl`. |
| `BASE_URL` | _empty_ | Overrides constructed `baseUrl` (HTTPS perimeter). |
| `META_CORE_DB_PATH` | `<META_CORE_PATH>/db/meta-core.db` | SQLite database file. Dev stacks set `/meta-core-db/meta-core.db` (named volume — SQLite WAL is unsafe on the 9p `/meta-core` bind of Windows dev boxes). |
| `META_CORE_DB_PATH` | `<META_CORE_PATH>/db/meta-core.db` | Truth database file (backed up). Dev stacks set `/meta-core-db/meta-core.db` (named volume — SQLite WAL is unsafe on the 9p `/meta-core` bind of Windows dev boxes). |
| `META_CORE_CACHE_PATH` | `/meta-core/db-cache/cache.db` | Cache database file (never backed up; rebuilt from truth when missing). Dev stacks keep it on a named volume too; store apps put the cache folder in Maison's `backup.exclude`. |
| `META_CORE_DB_CACHE_MB` | `64` | Page cache of the writer connection. |
| `META_CORE_DB_READER_CACHE_MB` | `16` | Page cache per reader connection. |
| `META_CORE_DB_READERS` | `8` | Reader connection pool size. |
@@ -547,7 +756,8 @@ via `/health`; metadata I/O and SSE event streams stop.
|---|---|
| `locks/kv-leader.lock` | Flock target. |
| `locks/kv-leader.info` | Leader API URL (plaintext). |
| `db/meta-core.db` (+ `-wal`, `-shm`) | Metadata store (SQLite). Replaced `db/redis/` in 2026-10. |
| `db/meta-core.db` (+ `-wal`, `-shm`) | Truth database (claims, principals, `cid_group`). Backed up. Replaced `db/redis/` in 2026-10; holds the claims schema since v2.0.0. |
| `db-cache/cache.db` (+ `-wal`, `-shm`) | Cache database (v1-layout `kv` table: merged view, lookup, counters, events, service scratch keys; field index, FTS, `claim_by_pid`, `local_file`). Never backed up. |
| `services/*.json` | Service registry. |
| `mounts/mounts.json` | rclone mount configuration. |
| `mounts/errors/{id}.error` | Mount error logs. |
@@ -561,7 +771,9 @@ via `/health`; metadata I/O and SSE event streams stop.
meta-core consolidates leader gating, metadata storage, event mirroring,
and file access for MetaMesh into one container with a typed HTTP+SSE
surface. The bash flock gate guarantees single-leader semantics without
an external consensus system. UUID-rooted storage with CID aliasing
treats all content hashes symmetrically and makes any-CID resolution
a single lookup. API-mediated access ensures no other service ever touches
an external consensus system. Metadata is a table of signed votes in a
backed-up truth database, merged by an apply loop into a rebuildable cache
that serves every read; meta-core verifies every claim and signs none.
CID groups behind a hidden local rid treat all content hashes
symmetrically. API-mediated access ensures no other service ever touches
the store.