568 lines
24 KiB
Markdown
568 lines
24 KiB
Markdown
# Meta-Core Architecture
|
|
|
|
## Document Purpose
|
|
|
|
This document describes the architectural design of **meta-core**, the
|
|
standalone container that owns the metadata store (an embedded SQLite
|
|
file; Redis until 2026-10), the leader-info file, the HTTP+SSE
|
|
metadata surface, and a WebDAV file server. It covers the leader gate,
|
|
storage data model, API design, the SSE event surface, and the integration
|
|
contract with other MetaMesh services.
|
|
|
|
For the on-disk key layout, see
|
|
[metadata-storage-structure.md](metadata-storage-structure.md). For the
|
|
external-access design (store lockdown, SSE wire contract, auth rules),
|
|
see [api-mediated-access.md](api-mediated-access.md). For the UUID-rooted
|
|
storage rationale, see [uuid-rooted-metadata.md](uuid-rooted-metadata.md).
|
|
|
|
---
|
|
|
|
## System Overview
|
|
|
|
MetaMesh is a collection of services that need a shared, ordered, durable
|
|
view of file metadata:
|
|
|
|
| Service | Language | Role |
|
|
|---|---|---|
|
|
| **meta-sort** | TypeScript | Watches folders, runs plugins, writes metadata via meta-core's HTTP API. |
|
|
| **meta-fuse** | TypeScript / Rust | Serves the virtual filesystem from metadata. |
|
|
| **meta-stremio** | Python | Stremio addon for HLS streaming. |
|
|
| **meta-dup** | TypeScript | Duplicate detection. |
|
|
| **meta-share** | Rust | Decentralised metadata sharing (libp2p / Kamilata). |
|
|
|
|
meta-core sits below all of these as the single owner of:
|
|
|
|
1. **The metadata store** — an embedded SQLite file
|
|
(`/meta-core/db/meta-core.db`) holding metadata and the event streams.
|
|
2. **The leader-info file** — the one place services look to discover meta-core's URLs.
|
|
3. **The HTTP+SSE surface** — the only way external services touch the data plane.
|
|
4. **A WebDAV server** — file access for plugins and other services.
|
|
|
|
### Core design principle
|
|
|
|
> External services never touch the store. Every read, write, and event flows
|
|
> through meta-core's HTTP+SSE surface.
|
|
|
|
This is the contract that PR D of [api-mediated-access.md](api-mediated-access.md)
|
|
landed and that the rest of this document assumes.
|
|
|
|
---
|
|
|
|
## Architecture
|
|
|
|
### Standalone container topology
|
|
|
|
meta-core runs as a dedicated container (`metacore-app`). The dev compose
|
|
runs a single instance; the legacy multi-instance topology (sibling
|
|
followers) was deprecated. The flock primitive is retained as the
|
|
correctness anchor — it guarantees that at most one process writes the
|
|
database file and the writable state under `/meta-core`.
|
|
|
|
```
|
|
┌────────────────────────────────────────────────────────────────────────┐
|
|
│ metacore-app (standalone container) │
|
|
│ │
|
|
│ supervisord │
|
|
│ ├─ leader-election.sh (flock → write kv-leader.info → exec self) │
|
|
│ ├─ nginx (TLS termination, proxy_cache for /webdav) │
|
|
│ ├─ rclone rcd (mount daemon) │
|
|
│ ├─ mount-watcher.sh (rclone mount lifecycle) │
|
|
│ └─ meta-core (Go) ──► HTTP + SSE + WebDAV on :9000 │
|
|
│ + embedded SQLite (db/meta-core.db) │
|
|
└────────────────────────────────────────────────────────────────────────┘
|
|
│
|
|
┌────────────────────────────┴────────────────────────────┐
|
|
│ /meta-core (shared volume) │
|
|
│ locks/kv-leader.{lock,info} │
|
|
│ db/meta-core.db (+ -wal, -shm) │
|
|
│ services/*.json │
|
|
│ mounts/{mounts.json, errors/} │
|
|
│ watchers.json │
|
|
│ cache/ (nginx proxy_cache) │
|
|
└─────────────────────────────────────────────────────────┘
|
|
▲
|
|
┌────────────────────────────┼────────────────────────────┐
|
|
│ │ │
|
|
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
|
│ meta-sort │ HTTP / SSE │ meta-fuse │ HTTP / SSE │ meta-stremio│
|
|
│ ├────────────►│ ├─────────────►│ │
|
|
│ │ │ │ │ │
|
|
└─────────────┘ └─────────────┘ └─────────────┘
|
|
```
|
|
|
|
### `/files` shared volume
|
|
|
|
Independently of `/meta-core`, every service mounts the `/files` volume
|
|
read-only (write access scoped to specific subtrees: `/files/plugin`,
|
|
`/files/corn` mounts). meta-core exposes this volume over WebDAV at
|
|
`/webdav/*` so plugins running in their own containers don't have to mount
|
|
it themselves.
|
|
|
|
```
|
|
/files
|
|
├── watch/ (host media, read-only)
|
|
├── test/ (test fixtures)
|
|
├── plugin/ (plugin output, read+write)
|
|
└── corn/ (rclone-managed mounts)
|
|
```
|
|
|
|
---
|
|
|
|
## Leader Gate
|
|
|
|
### Mechanism: bash + flock
|
|
|
|
Leader election is implemented in `docker/leader-election.sh`, not in Go.
|
|
The script:
|
|
|
|
1. Opens `${META_CORE_PATH}/locks/kv-leader.lock` on file descriptor 200.
|
|
2. Tries `flock -n 200`. On success it transitions to LEADER, writes the
|
|
API URL to `${META_CORE_PATH}/locks/kv-leader.info`, and `exec`s
|
|
supervisord — replacing itself in the process tree. When supervisord
|
|
exits, the kernel releases the flock automatically.
|
|
3. On failure (someone else holds the lock) it sleeps `ELECTION_RETRY_SECS`
|
|
(default `5s`) and retries. Followers loop in bash and never start the
|
|
Go binary.
|
|
|
|
```
|
|
Container startup
|
|
│
|
|
▼
|
|
┌──────────────────┐
|
|
│ leader-election │
|
|
│ .sh │
|
|
│ flock(kv-leader │
|
|
│ .lock) │
|
|
└────────┬─────────┘
|
|
│
|
|
┌────┴────┐
|
|
│ │
|
|
acquired blocked
|
|
│ │
|
|
▼ ▼
|
|
┌─────────┐ ┌──────────────┐
|
|
│ Write │ │ sleep N, │
|
|
│ kv- │ │ retry │
|
|
│ leader │ └──────────────┘
|
|
│ .info │
|
|
│ │
|
|
│ exec │
|
|
│ super- │
|
|
│ visord │
|
|
└─────────┘
|
|
│
|
|
▼
|
|
┌──────────────────────────────┐
|
|
│ supervisord starts: │
|
|
│ nginx → rclone → │
|
|
│ meta-core Go binary │
|
|
└──────────────────────────────┘
|
|
```
|
|
|
|
### Implications for the Go binary
|
|
|
|
The Go binary at `cmd/meta-core/main.go` is invoked only after the bash
|
|
gate wins. Therefore:
|
|
|
|
- `internal/leader/election.go` does **not** run an election loop. It
|
|
contains `LeaderInfoProvider`, which builds the leader info struct from
|
|
local hostname + IP + config and returns it on demand. There are no
|
|
`OnBecomeLeader` / `OnBecomeFollower` / `OnLeaderLost` callbacks; those
|
|
states do not exist inside the Go process.
|
|
- `LeaderLockInfo.RedisUrl` is always empty in the response from
|
|
`/urls` (kept for wire compatibility). PR D of
|
|
api-mediated-access.md removed direct Redis exposure, and since 2026-10
|
|
there is no Redis at all; consumers route metadata I/O through
|
|
meta-core's HTTP API and SSE event streams.
|
|
|
|
### Lock-info file
|
|
|
|
`/meta-core/locks/kv-leader.info` is a single plaintext line:
|
|
|
|
```
|
|
http://10.0.1.50:9000
|
|
```
|
|
|
|
Other services read this file, then call `GET /urls` for full discovery
|
|
(hostname, baseUrl, apiUrl, webdavUrl, webdavUrlInternal).
|
|
|
|
### Failure handling
|
|
|
|
- **meta-core process dies** → supervisord restarts it. The bash flock
|
|
loop does not need to participate — supervisord owns the lock through
|
|
its lifetime.
|
|
- **Container dies** → kernel releases the flock. If a standby
|
|
`metacore-app` is provisioned, its bash loop will acquire on next tick.
|
|
The dev compose runs a single instance; failover is "restart the
|
|
container."
|
|
- **The store is in-process** — there is no separate database process to
|
|
crash. A meta-core crash is a process restart; SQLite's WAL recovers the
|
|
last committed transaction on reopen.
|
|
|
|
---
|
|
|
|
## Schema-Version Gate
|
|
|
|
Before serving any traffic, `cmd/meta-core/main.go:54-61` calls
|
|
`storage.EnsureSchemaVersion`. The sentinel lives in the store at
|
|
`meta-core:schema-version` (`internal/storage/schema_version.go`). The
|
|
gate refuses to boot when the existing store layout predates the
|
|
UUID-rooted schema, so legacy `file:midhash256:*` data can't leak into
|
|
the current code paths. In alpha there is no automated migration —
|
|
operators wipe with `pnpm run clean:all` and restart.
|
|
|
|
---
|
|
|
|
## Data Model
|
|
|
|
### Two concepts: metadata vs. file bytes
|
|
|
|
```
|
|
┌────────────────────────────────────────────────────────────────────┐
|
|
│ METADATA (embedded SQLite, kv table) │
|
|
│ │
|
|
│ Root: opaque UUIDv7 (Crockford Base32; ULID layout, 26 chars) │
|
|
│ Storage: flat string keys │
|
|
│ file:<uuid>/<property> → string value │
|
|
│ file:<uuid>/cids/<bareCid> → "true" (key-set) │
|
|
│ dup:<uuid>/<path> → "" (one per path) │
|
|
│ │
|
|
│ No stored canonical_cid — derived by rank on read. │
|
|
│ │
|
|
│ Aliases (reverse index, one key each): │
|
|
│ cid:<bareCid> → <uuid> │
|
|
│ │
|
|
│ Index: idx:roots/<uuid> → "" (one per root) │
|
|
│ │
|
|
│ Mintable without a local file (remote / deleted entries OK). │
|
|
│ Resolvable by any registered CID in O(1). │
|
|
└────────────────────────────────────────────────────────────────────┘
|
|
|
|
┌────────────────────────────────────────────────────────────────────┐
|
|
│ FILE BYTES │
|
|
│ │
|
|
│ Location: /files volume (shared across containers). │
|
|
│ Access: path resolution via metadata, served via /webdav/*, │
|
|
│ /file/{cid} (reverse-lookup), or /data/{hash}. │
|
|
│ Path stored as filePath in the metadata, relative to /files. │
|
|
└────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
### Why UUID-rooted?
|
|
|
|
Roots were previously content-hash tokens (`file:midhash256:abc/...`),
|
|
which privileged one hash algorithm. UUID-rooted storage treats every CID
|
|
symmetrically as an alias, makes `GET /api/meta/<cid>` O(1) for any
|
|
registered CID, and lets new digest types be added without rewriting root
|
|
keys. See [uuid-rooted-metadata.md](uuid-rooted-metadata.md) for the
|
|
design rationale and [metadata-storage-structure.md](metadata-storage-structure.md)
|
|
for the on-disk layout.
|
|
|
|
### Path resolution
|
|
|
|
`filePath` values stored in metadata are absolute paths under the `/files`
|
|
volume (e.g. `/files/watch/movies/Movie.2024.mkv`). The `/files` prefix is
|
|
itself a per-host detail — moving the volume only requires re-mounting it
|
|
at the same path inside the container.
|
|
|
|
---
|
|
|
|
## API Design
|
|
|
|
### Endpoint catalogue
|
|
|
|
The router is in `internal/api/server.go`. Endpoint set, by surface:
|
|
|
|
```
|
|
Bootstrap / discovery
|
|
GET /health
|
|
GET /status
|
|
GET /leader
|
|
GET /urls
|
|
GET /neighbors (alias: /api/neighbors)
|
|
GET /api/stats
|
|
|
|
Metadata (primary surface for other services)
|
|
GET /meta
|
|
GET /meta/{hash}
|
|
PUT /meta/{hash}
|
|
PATCH /meta/{hash}
|
|
DELETE /meta/{hash}
|
|
GET /meta/{hash}/{key...}
|
|
PUT /meta/{hash}/{key...}
|
|
DELETE /meta/{hash}/{key...}
|
|
POST /meta/{hash}/_add/{key...}
|
|
|
|
CID-addressed (public, auth-bypassed at the perimeter)
|
|
GET /api/meta/{cid}
|
|
GET /api/file/{cid}/info
|
|
GET /file/{cid}
|
|
POST /file/cid
|
|
HEAD /data/{hash}
|
|
GET /data/{hash}/path
|
|
|
|
Editor / KV / schema / snapshot
|
|
GET /api/metadata/hash-ids
|
|
GET /api/files/tuples
|
|
GET /api/metadata/list
|
|
POST /api/metadata/search
|
|
POST /api/metadata/batch
|
|
POST /api/metadata/clear
|
|
{GET,PUT,DELETE} /api/metadata/{hashId}
|
|
{GET,PUT} /api/metadata/{hashId}/property
|
|
GET /api/kv/{info,keys,tree,search,find,value}
|
|
PUT /api/kv/value
|
|
DELETE /api/kv/value
|
|
GET /api/kv/key/{key...}
|
|
GET /api/schema
|
|
POST /api/schema/rescan
|
|
GET /api/snapshot/export
|
|
POST /api/snapshot/import
|
|
POST /api/snapshot/wipe
|
|
|
|
SSE event streams (HTTP-mirror of the store's event streams)
|
|
GET /api/events/files
|
|
GET /api/events/meta
|
|
|
|
Mounts (rclone-only)
|
|
GET /api/mounts
|
|
POST /api/mounts
|
|
GET /api/mounts/{id}
|
|
PUT /api/mounts/{id} (also PATCH)
|
|
DELETE /api/mounts/{id}
|
|
POST /api/mounts/{id}/{mount,unmount,safe-unmount,scan}
|
|
GET /api/mounts/rclone/remotes
|
|
|
|
Watchers
|
|
{GET,POST} /api/watchers
|
|
{GET,PUT,DELETE} /api/watchers/{id}
|
|
POST /api/watchers/{id}/{scan,reset}
|
|
POST /api/watchers/{scan-all,reset-all}
|
|
(deprecated: /api/scan/trigger, /api/scan/status — see watcher/handlers.go:32)
|
|
|
|
(The one-shot Redis-era admin migrations — `/api/admin/migrate-dual-roots`,
|
|
`migrate-domain-screen`, `migrate-user-cids`, `sweep-literature-echoes` —
|
|
were removed with the 2026-10 SQLite switch; they had been run once on
|
|
watch.nsl.sh beforehand.)
|
|
|
|
WebDAV
|
|
/webdav/... (GET, PUT, DELETE, MKCOL, COPY, MOVE, PROPFIND)
|
|
```
|
|
|
|
No `/metrics` endpoint is exposed. Observability is via container logs +
|
|
the `/health` and `/status` JSON.
|
|
|
|
### Auth perimeter
|
|
|
|
In the dev and prod stacks Caddy + an nginx-hash-lock sidecar enforce
|
|
OIDC auth in front of meta-core. The following paths bypass auth (so
|
|
peers and unauthenticated tooling can reach them):
|
|
|
|
- `/health`
|
|
- `/webdav/*`
|
|
- `/meta/*`
|
|
- `/file/cid`
|
|
- `/api/file/{cid}` and `/api/meta/{cid}`
|
|
- (`/api/events/files`, `/api/events/meta` are also auth-bypassed but
|
|
**should not** be exposed publicly via Caddy; they're inside-only.)
|
|
|
|
See `docs/api-mediated-access.md` "Auth" for the rationale.
|
|
|
|
### SSE event streams — wire contract
|
|
|
|
`/api/events/files` and `/api/events/meta` are SSE mirrors of the
|
|
`file:events` and `meta:events` streams, stored as `ev:<stream>/…` rows of
|
|
the kv table (`internal/kv/stream.go`). The contract
|
|
(`internal/api/sse_events.go`):
|
|
|
|
- One SSE event per stream entry.
|
|
- `id:` is the opaque `<ms>-<seq>` entry ID (the Redis Stream form, kept so
|
|
cursors persisted before the 2026-10 switch still compare) — clients echo
|
|
it back via `Last-Event-ID` on reconnect. Without `Last-Event-ID` the
|
|
stream starts at the head (resolved once at connect); `0` / `0-0` replay
|
|
from the beginning.
|
|
- `event:` is the `type` field of the underlying entry.
|
|
- `data:` is the JSON of the rest of the entry.
|
|
- Heartbeats: SSE comment `:keep-alive\n\n` every 30s of silence.
|
|
- On reconnect with `Last-Event-ID`, the handler resumes from the next
|
|
entry. If retention has trimmed past the cursor, the handler emits one
|
|
`event: gap` payload before resuming from the oldest available entry.
|
|
|
|
External services use SSE with `Last-Event-ID` as the resume primitive.
|
|
`meta:events` entries are appended by the storage engine in the same
|
|
transaction as the change they describe; the Redis-era keyspace-notification
|
|
hop through `MetaPublisher` is gone.
|
|
|
|
### WebDAV URLs
|
|
|
|
`/urls` exposes two WebDAV URLs (`internal/leader/election.go`):
|
|
|
|
| Field | Purpose | Construction |
|
|
|---|---|---|
|
|
| `webdavUrl` | External access | `{BASE_URL}/webdav` (or `http://{ip}:{API_PORT}/webdav` if `BASE_URL` is unset) |
|
|
| `webdavUrlInternal` | Container-to-container | `http://{hostname}:{META_CORE_HTTP_PORT}/webdav` |
|
|
|
|
Container plugins should use `webdavUrlInternal`. Browsers / external
|
|
clients use `webdavUrl` (which goes through nginx + Caddy).
|
|
|
|
WebDAV caching is handled by nginx's `proxy_cache` (see
|
|
`docker/nginx.conf`), not by a Go cache layer. There is no
|
|
`internal/cache` package.
|
|
|
|
---
|
|
|
|
## Implementation
|
|
|
|
### Why Go
|
|
|
|
| Requirement | Go advantage |
|
|
|---|---|
|
|
| Language-agnostic | HTTP API, no runtime dependency on callers |
|
|
| Single binary | Static compilation |
|
|
| Low resource | ~10-20 MB resident |
|
|
| HTTP server | stdlib `net/http` + gorilla/mux |
|
|
| Metadata store | embedded SQLite via `modernc.org/sqlite` (pure Go, CGO off) |
|
|
| WebDAV | `golang.org/x/net/webdav` |
|
|
|
|
### Repo layout
|
|
|
|
```
|
|
packages/meta-core/
|
|
├── cmd/meta-core/main.go entry point
|
|
├── internal/
|
|
│ ├── api/ HTTP server + handlers + SSE
|
|
│ ├── cid/ CID parsing + algorithm ranking
|
|
│ ├── config/ env-driven configuration
|
|
│ ├── discovery/ service registry + dead-service cleaner
|
|
│ ├── events/ boot-time meta:events republish
|
|
│ ├── kv/ SQLite kv engine + event streams
|
|
│ ├── leader/ LeaderInfoProvider (no election logic)
|
|
│ ├── mounts/ rclone mount manager + handlers + stats
|
|
│ ├── schema/ live per-field schema indexer
|
|
│ ├── snapshot/ snapshot export / import / wipe
|
|
│ ├── storage/ flat-key records + search + UUID + CID reverse index + schema gate
|
|
│ ├── watcher/ scanner + dispatcher + midhash + state
|
|
│ ├── watchers/ polling-based watcher configs
|
|
│ └── webdav/ /webdav/* handler
|
|
├── docker/
|
|
│ ├── entrypoint.sh
|
|
│ ├── leader-election.sh the actual leader election
|
|
│ ├── mount-watcher.sh
|
|
│ ├── nginx.conf
|
|
│ └── supervisord.conf
|
|
├── editor/ React metadata browser (Vite)
|
|
├── dashboard/ React status dashboard (Vite)
|
|
├── docs/
|
|
└── go.mod
|
|
```
|
|
|
|
### Startup sequence (`cmd/meta-core/main.go`)
|
|
|
|
```
|
|
1. Load configuration from environment
|
|
2. Build LeaderInfoProvider (no election; bash gate already won)
|
|
3. Open the SQLite store (META_CORE_DB_PATH or /meta-core/db/meta-core.db)
|
|
4. EnsureSchemaVersion (abort if the store layout is stale)
|
|
5. Start service discovery + dead-service cleaner
|
|
6. NewServer + Server.Start (boots watcher, mounts, WebDAV, API)
|
|
7. On first connection: start MetaPublisher + schema Indexer,
|
|
republish existing metadata onto meta:events
|
|
8. Wait for SIGINT/SIGTERM
|
|
9. Shutdown in reverse order
|
|
```
|
|
|
|
### Process management
|
|
|
|
There is no database process: the store is embedded in the Go binary
|
|
(`docker/supervisord.conf` runs meta-core, nginx and rclone). Until
|
|
2026-10 supervisord also ran `redis-server` on `localhost:6379`.
|
|
|
|
---
|
|
|
|
## Failure Modes
|
|
|
|
### Container restart
|
|
|
|
Supervisord restarts crashed processes (meta-core, nginx, rclone)
|
|
individually. The bash flock is held by supervisord's process tree, so
|
|
internal crashes don't trigger re-election.
|
|
|
|
### Store durability
|
|
|
|
The SQLite file runs in WAL mode with `synchronous=NORMAL`: every
|
|
committed transaction survives a meta-core process crash. A record write — fields, root index, CID aliases,
|
|
FTS row and its events — is one transaction, so a crash never leaves a
|
|
half-written record.
|
|
|
|
### Container crash
|
|
|
|
Kernel releases the flock; a standby container (if any) can take over.
|
|
The dev compose runs a single instance, so the recovery path is "restart
|
|
the container."
|
|
|
|
### Schema mismatch
|
|
|
|
`EnsureSchemaVersion` refuses to boot. Operators wipe with
|
|
`pnpm run clean:all` (or equivalent) and restart. This is an alpha-stage
|
|
hard-fail; there is no automated migration path.
|
|
|
|
### Shared volume unavailable
|
|
|
|
All meta-core operations fail. Other services see meta-core as unhealthy
|
|
via `/health`; metadata I/O and SSE event streams stop.
|
|
|
|
---
|
|
|
|
## Configuration Reference
|
|
|
|
### Environment variables
|
|
|
|
| Variable | Default | Description |
|
|
|---|---|---|
|
|
| `META_CORE_PATH` | `/meta-core` | Shared volume root. |
|
|
| `FILES_PATH` | `/files` | Files volume root. |
|
|
| `SERVICE_NAME` | `meta-core` | Identity in the service registry. |
|
|
| `SERVICE_VERSION` | `1.0.0` | Reported in `/status`. |
|
|
| `API_PORT` | `8180` | External port baked into `baseUrl`. |
|
|
| `BASE_URL` | _empty_ | Overrides constructed `baseUrl` (HTTPS perimeter). |
|
|
| `META_CORE_DB_PATH` | `<META_CORE_PATH>/db/meta-core.db` | SQLite database file. Dev stacks set `/meta-core-db/meta-core.db` (named volume — SQLite WAL is unsafe on the 9p `/meta-core` bind of Windows dev boxes). |
|
|
| `META_CORE_DB_CACHE_MB` | `64` | Page cache of the writer connection. |
|
|
| `META_CORE_DB_READER_CACHE_MB` | `16` | Page cache per reader connection. |
|
|
| `META_CORE_DB_READERS` | `8` | Reader connection pool size. |
|
|
| `META_CORE_HTTP_PORT` | `9000` | Go HTTP+SSE+WebDAV port. |
|
|
| `META_CORE_HTTP_HOST` | `0.0.0.0` | HTTP bind address. |
|
|
| `HEALTH_CHECK_INTERVAL_MS` | `5000` | Internal health-loop cadence. |
|
|
| `HEARTBEAT_INTERVAL_MS` | `30000` | Service registry heartbeat. |
|
|
| `STALE_THRESHOLD_MS` | `60000` | Age past which a registry entry is stale. |
|
|
| `CLEANUP_INTERVAL_MS` | `600000` | Dead-service cleaner cadence. |
|
|
| `DEAD_SERVICE_THRESHOLD_MS` | `600000` | Age past which a stale entry is deleted. |
|
|
| `WATCH_INTERVAL_MS` | `1000` | Watcher poll cadence. |
|
|
| `DEBOUNCE_MS` | `30000` | File-event debounce window. |
|
|
| `ENABLE_FILE_WATCHER` | `true` | Disable to suppress the in-process watcher. |
|
|
| `ELECTION_RETRY_SECS` | `5` | Bash flock retry interval (consumed by `docker/leader-election.sh`, not the Go binary). |
|
|
|
|
### File paths under `/meta-core`
|
|
|
|
| Path | Purpose |
|
|
|---|---|
|
|
| `locks/kv-leader.lock` | Flock target. |
|
|
| `locks/kv-leader.info` | Leader API URL (plaintext). |
|
|
| `db/meta-core.db` (+ `-wal`, `-shm`) | Metadata store (SQLite). Replaced `db/redis/` in 2026-10. |
|
|
| `services/*.json` | Service registry. |
|
|
| `mounts/mounts.json` | rclone mount configuration. |
|
|
| `mounts/errors/{id}.error` | Mount error logs. |
|
|
| `watchers.json` | Polling watcher configuration. |
|
|
| `cache/` | nginx `proxy_cache` directory. |
|
|
|
|
---
|
|
|
|
## Summary
|
|
|
|
meta-core consolidates leader gating, metadata storage, event mirroring,
|
|
and file access for MetaMesh into one container with a typed HTTP+SSE
|
|
surface. The bash flock gate guarantees single-leader semantics without
|
|
an external consensus system. UUID-rooted storage with CID aliasing
|
|
treats all content hashes symmetrically and makes any-CID resolution
|
|
a single lookup. API-mediated access ensures no other service ever touches
|
|
the store.
|