perf improved hash plugin

docs/midhash256.md

@@ -0,0 +1,80 @@
+# MidHash256
+
+## Purpose
+
+O(1) file identification hash - constant time regardless of file size. Designed for quick duplicate detection and large file identification before expensive full-hash operations.
+
+## Multicodec
+
+- **Code**: `0x1000` (custom, not in official multicodec registry)
+- **CID codec**: `0x55` (raw)
+- **Output key**: `cid_midhash256`
+
+## Algorithm
+
+```
+1. Read file size (S)
+2. If S <= 1MB:
+     sample = entire file content
+3. If S > 1MB:
+     seek to (S - 1MB) / 2
+     sample = read 1MB from middle
+4. Hash = SHA-256(S as u64 big-endian || sample)
+```
+
+### Pseudocode
+
+```rust
+fn midhash256(path: &Path) -> [u8; 32] {
+    let size = file_size(path);
+    let mut hasher = SHA256::new();
+
+    // Always include size as 8-byte big-endian prefix
+    hasher.update(size.to_be_bytes());
+
+    if size <= 1_048_576 {
+        // Small file: hash entire content
+        hasher.update(read_all(path));
+    } else {
+        // Large file: hash middle 1MB
+        let middle_start = (size - 1_048_576) / 2;
+        hasher.update(read_at(path, middle_start, 1_048_576));
+    }
+
+    hasher.finalize()
+}
+```
+
+## Properties
+
+| Property | Value |
+|----------|-------|
+| Time complexity | O(1) for files > 1MB |
+| Space complexity | O(1) - fixed 1MB buffer |
+| Hash size | 32 bytes (SHA-256) |
+| Collision resistance | Same as SHA-256 for middle content |
+
+## Use Cases
+
+1. **Quick duplicate detection** - Identify potential duplicates before computing full hashes
+2. **Large file identification** - Identify terabyte files in ~20ms regardless of size
+3. **Pre-filtering** - Filter candidates before expensive operations (full hash, content analysis)
+4. **Change detection** - Quick check if middle content changed (useful for video files)
+
+## Limitations
+
+- Files differing only at edges (first/last 512KB for 2MB+ files) will have same midhash
+- Not suitable as a standalone deduplication hash - use as pre-filter only
+- Custom multicodec code may conflict with future official allocations
+
+## CID Example
+
+```
+File: movie.mkv (4.7 GB)
+MidHash256: 3a7b...c2d1 (32 bytes)
+CID: bafkreibh4xvl5... (base32lower encoded CIDv1)
+```
+
+## Integration
+
+The hash is computed separately from the standard single-pass algorithms since it requires seeking to the middle of the file. For files <= 1MB, it reads the entire file like other algorithms.