README: sync with current code

README.md

@@ -1,31 +1,38 @@
 # meta-dup
 
 Standalone duplicate detection service for MetaMesh v2. Detects duplicate files by:
-- **Hash (SHA-256)**: Files with identical content
+- **Hash**: Files with identical content — same `midhash256` (record address) seen at several paths, or same full-file SHA-256
-- **Title**: Files with the same parsed title (from filename patterns)
+- **Title**: Files with the same parsed title (+ year / season / episode when known)
+
+> **Status: not wired into the current stack.** meta-dup was removed from the main dev compose (`dev/docker-compose.yml`) in March 2026 and has no MetaAppStore app. Its storage layer still talks to Redis directly (ioredis + Redis Streams), while meta-core has since stopped publishing a `redisUrl` (api-mediated-access lockdown) — so against a current meta-core it has no Redis to connect to. meta-fuse shows the migrated pattern (HTTP reads via `/meta/{hash}*` + SSE `/api/events/meta`). Treat the sections below as a description of the code, not of a running service.
 
 ## Architecture
 
 meta-dup is a read-only service that:
-1. Subscribes to Redis Streams (`meta-sort:events`) for real-time updates
+1. Locates meta-core over UDP multicast (meta-discovery v1, `239.255.77.1:9399`) — or uses `META_CORE_URL` when pinned
-2. Maintains an in-memory index of duplicates
+2. Builds an in-memory duplicate index from every record in Redis on startup
-3. Provides REST API and web dashboard
+3. Consumes Redis Streams for real-time updates (`file:events` for file add/change/delete/rename, `meta:events` for title-field changes)
+4. Provides a REST API and web dashboard (nginx on port 80 → Fastify on 3000)
 
 ```
-meta-sort ──► Redis Streams ──► meta-dup ──► Dashboard (port 8183)
+meta-core ──► Redis Streams ──► meta-dup ──► Dashboard / REST API
-              (events)          (consumer)
+              (file:events,     (consumer)
+               meta:events)
 ```
 
+See [service-discovery.md](../../docs/project-architecture/service-discovery.md) for the discovery protocol.
+
 ## API Endpoints
 
 | Endpoint | Method | Description |
 |----------|--------|-------------|
-| `/health` | GET | Health check with stats |
+| `/health` | GET | Health check with stats (`status: ok\|degraded`) |
 | `/api/duplicates` | GET | All duplicates (hash + title) |
 | `/api/duplicates/hash` | GET | Hash duplicates only |
 | `/api/duplicates/title` | GET | Title duplicates only |
 | `/api/duplicates/stats` | GET | Duplicate statistics |
 | `/api/duplicates/rebuild` | POST | Force rebuild from Redis |
+| `/api/neighbors` | GET | Services heard over meta-discovery v1 (dashboard nav menu) |
 
 ## Response Format
 
@@ -33,7 +40,7 @@ meta-sort ──► Redis Streams ──► meta-dup ──► Dashboard (port 8
 {
   "hashDuplicates": [
     {
-      "key": "sha256-hash",
+      "key": "<midhash256 or sha256>",
       "files": [
         { "hashId": "...", "filePath": "/files/...", "sizeByte": 123456 }
       ]
@@ -42,10 +49,13 @@ meta-sort ──► Redis Streams ──► meta-dup ──► Dashboard (port 8
   "titleDuplicates": [...],
   "stats": {
     "hashGroupCount": 5,
+    "hashFileCount": 12,
     "titleGroupCount": 3,
-    "totalFilesTracked": 100
+    "titleFileCount": 7,
+    "totalFilesTracked": 100,
+    "lastUpdated": "2026-01-01T00:00:00.000Z"
   },
-  "computedAt": 1234567890
+  "computedAt": "2026-01-01T00:00:00.000Z"
 }
 ```
 
@@ -58,34 +68,35 @@ pnpm install
 pnpm build
 ```
 
-### Run (in Docker dev environment)
+### Docker image
 ```bash
-cd dev
+docker build -t meta-dup .
-./scripts/start.sh  # Starts all services including meta-dup
+# The image also copies the meta-core binary from META_CORE_IMAGE
+# (default ghcr.io/worph/meta-core:latest) and runs it under supervisord.
 ```
 
-### Rebuild
+CI publishes `ghcr.io/worph/meta-dup` on `v*` tags (`.github/workflows/docker-publish.yml`).
-```bash
-cd dev
-./scripts/reload-meta-dup.sh
-```
 
-### Tests
+### Dev stack / tests
-```bash
+The main dev stack no longer defines a meta-dup container, so `dev/scripts/reload-meta-dup.sh` (targets container `meta-dup`) and the `dup` suite of `dev/test/test.sh` have nothing to run against until it is re-added.
-docker exec meta-test-runner /app/test/test.sh dup
-```
 
 ## Configuration
 
-meta-dup uses service discovery via `/meta-core/locks/kv-leader.info` to find the Redis instance. No additional configuration required.
+meta-core is located over UDP (meta-discovery v1); no `/meta-core` volume mount or `REDIS_URL` is needed.
 
 ### Environment Variables
 
 | Variable | Default | Description |
 |----------|---------|-------------|
-| `BASE_URL` | `http://localhost:8183` | External URL for service |
+| `META_CORE_URL` | — | Pin meta-core's API URL; UDP discovery never overrides it |
-| `SERVICE_NAME` | `meta-dup` | Service identifier |
+| `ENABLE_UDP_DISCOVERY` | on | Set `false`/`0` to disable meta-discovery |
+| `BASE_URL` | — | External URL (announced for the nav menu) |
+| `PUBLIC_URL` | — | Announced URL; wins over `BASE_URL` (debug-direct, no Caddy) |
 | `API_PORT` | `3000` | Internal API port (nginx proxies to 80) |
+| `API_HOST` | `0.0.0.0` | API bind address |
+| `REDIS_PREFIX` | `''` | Key prefix in Redis |
+| `FILES_VOLUME` | `/files` | Prepended to the relative `filePath` of each record |
+| `ALLOW_LEGACY_REDIS_URL` | — | `1` downgrades the "meta-core still publishes redisUrl" startup error to a warning |
 
 ## Package Structure
 
@@ -93,14 +104,19 @@ meta-dup uses service discovery via `/meta-core/locks/kv-leader.info` to find th
 packages/meta-dup/
 ├── package.json            # Workspace root
 ├── pnpm-workspace.yaml
+├── Dockerfile
+├── docker/                 # nginx.conf, supervisord.conf
 └── packages/
     ├── meta-dup-core/      # Backend service
     │   ├── src/
-    │   │   ├── index.ts           # Entry point
+    │   │   ├── index.ts              # Entry point, file:events consumer
-    │   │   ├── DuplicateIndex.ts  # Core duplicate detection
+    │   │   ├── DuplicateIndex.ts     # Core duplicate detection
+    │   │   ├── MetaEventConsumer.ts  # meta:events consumer (title updates)
     │   │   ├── api/
-    │   │   │   └── APIServer.ts   # Fastify REST API
+    │   │   │   └── APIServer.ts      # Fastify REST API
-    │   │   └── kv/                # Redis client & leader discovery
+    │   │   ├── discovery/
+    │   │   │   └── meshdisco.ts      # meta-discovery v1 (mirrored, see scripts/check-mirrors.sh)
+    │   │   └── kv/                   # Redis client + meta-core locator (LeaderClient)
     │   └── package.json
     └── meta-dup-ui/        # React dashboard
         ├── src/
@@ -112,23 +128,21 @@ packages/meta-dup/
 ## How Duplicate Detection Works
 
 ### Hash Duplicates
-Files with the same `cid_sha2-256` (full file hash) are exact content duplicates.
+- **Path-based**: `file:events` add/change/rename messages carry `midhash256` + `path`; one midhash seen at several paths is a duplicate.
+- **Full hash**: the full-file SHA-256 is read from the record's bare-CID key-set (`cids/<cid>`, picking the sha2-256 multicodec). Records with the same SHA-256 are exact content duplicates. See [METADATA_KEYS.md](../../METADATA_KEYS.md).
 
 ### Title Duplicates
-Files parsed to the same title (e.g., "Movie Name (2020)") from their filename patterns. Useful for finding different quality versions of the same content.
+Files resolving to the same normalized key `title[:year][:sN][:eN]`. Useful for finding different quality versions of the same content. Title fields (`title`, `originalTitle`, `fileName`, `year`, `movieYear`, `season`, `episode`, `tmdb`) are re-read on `meta:events` changes, debounced 500 ms.
 
 ## Redis Streams Integration
 
-meta-dup subscribes to `meta-sort:events` stream with consumer group `meta-dup-consumer`:
+| Stream | Consumer group | Handles |
-
+|--------|----------------|---------|
-```
+| `file:events` | `meta-dup-consumer` | `add`, `change`, `delete`, `rename`; legacy `batch`, `reset`, `plugin:complete` |
-Event types:
+| `meta:events` | `meta-dup-meta-consumer` | `set`/`del` on `file:{hashId}/{field}` for title fields |
-- batch: File batch processed (add/update duplicates)
-- reset: Full index rebuild required
-```
 
 On startup, meta-dup:
-1. Creates/joins consumer group
+1. Rebuilds the full index from Redis (via the `file:__index__` set)
-2. Processes any pending entries
+2. Creates/joins the `file:events` consumer group
-3. Rebuilds full index from Redis `file:*` keys
+3. Processes pending entries idle > 30 s
-4. Starts listening for new events
+4. Starts listening for new events on both streams