replace Redis with an embedded SQLite string->string KV store (group commit, in-tx events, SQL search)
Build and Push Docker Image / build (push) Successful in 2m44s

This commit is contained in:
Worph
2026-10-19 20:51:00 +00:00
parent c53146e514
commit dc5cbdfb9b
80 changed files with 4601 additions and 6077 deletions
+9 -2
View File
@@ -20,6 +20,12 @@
> `cid:<bareCid>` → uuid. A CIDv1's multicodec already names its hash
> function, so the `<algo>:` prefix was redundant.
>
> 3. **The store is no longer Redis (2026-10).** The same keys now live in
> one string→string table of an embedded SQLite file; Redis SETs became
> one key per member (`file:__index__` → `idx:roots/<uuid>`,
> `file:<uuid>/duplicates` → `dup:<uuid>/<path>`). Redis types named
> below are the shape at the time.
>
> Text below that references either shape describes the *proposal*, not the
> code. The rank ordering and everything else did ship as written.
@@ -135,7 +141,7 @@ sole writer is meta-core itself:
| Key | Type (as shipped) | Purpose |
|-----|------|---------|
| `cids/<bareCid>` | STRING `"true"` — a *key-set*, one key per member | All CIDs known for this file, midhash included. Mirror of the `cid:*` reverse index. Source of truth for delete-time GC. **Proposed as a Redis SET of `<algo>:<value>` tokens; shipped as a key-set of bare CIDs** — a key-set merges conflict-free across peers and the multicodec already names the algorithm. |
| `duplicates` | Redis SET | Additional file paths whose content matches this entry. One UUID = one set of metadata; multiple physical files may share that content. Consumed by meta-dup. |
| `duplicates` | Redis SET at the time; since 2026-10 one `dup:<uuid>/<path>` key per path | Additional file paths whose content matches this entry. One UUID = one set of metadata; multiple physical files may share that content. Consumed by meta-dup. |
| ~~`canonical_cid`~~ | *(not stored)* | **Dropped.** Was to hold the externally-presented CID. Now derived on read by ranking the `cids` members; surfaced only on `GET /api/meta/{cid}`. |
Existing meta-core-owned fields (`filePath`, `size`, `mtime`, `midhash256`)
@@ -148,7 +154,8 @@ for using sets where they're the right structure.~~ **Reversed for `cids`.**
The all-strings convention won: `cids` ships as flat `cids/<cid>` STRING keys
so that two peers merging the same record can never disagree about set
membership. `duplicates` remained a Redis SET (it is leader-local and never
merged across peers).
merged across peers) until the 2026-10 SQLite switch, which flattened it to
`dup:<uuid>/<path>` keys.
### Canonical CID selection