docs: search plan + counters for v1.2.0; tools/search-parity (parity + latency vs a brute-force reference)

2026-10-19 21:59:36 +00:00
parent fb4bdad61b
commit 5369dc4b9c
5 changed files with 241 additions and 4 deletions
@@ -0,0 +1,29 @@
+# search-parity
+
+Regression check for `POST /api/metadata/search` against a running meta-core.
+
+- `parity.py <meta-core.db>` builds ~230 queries from real records (free text,
+  short tokens, key-set and flat filters, exact/contains, multi-filter). It
+  computes each query's expected result by brute force: one streaming pass
+  over every record, using a port of the Go matchers. Then it diffs that
+  against the live API. Any `lost` result means the SQL candidate superset
+  dropped a real match.
+- `timing.py [meta-core.db]` times the same query set at limit 100 and 2000.
+
+Run them inside the container. It has python3 and reads the database
+read-only:
+
+```bash
+docker cp tools/search-parity metacore-app:/tmp/sp
+docker exec metacore-app python3 -u /tmp/sp/parity.py /meta-core/db/meta-core.db
+docker exec metacore-app python3 -u /tmp/sp/timing.py
+```
+
+The reference pass takes ~10 min at 169k records. Records written while it
+runs can show up as `extra` at a limit-2000 boundary; check their UUIDv7 time
+before treating a diff as real.
+
+History: these tools gated the 2026-10 Redis → SQLite cut-over. Results:
+0/217 diffs on v1.1.0, and 232/234 on v1.2.0, where the 2 diffs were writes
+made during the run. See
+`docs/project-architecture/metacore-storage-engine.md` in the meta-root.