rework plugin

README.md

@@ -1 +1,138 @@
-# meta-feeder-sdk
+# meta-feeder-sdk
+
+The source-agnostic foundation shared by the MetaMesh gateway core and every
+feeder sidecar.
+
+A **feeder** bridges one external source into the MetaMesh network. It
+implements [`FeederPlugin`](src/plugin.rs) — find records, resolve them to
+content-addressed outcomes, optionally fetch bytes — and `serve_feeders` exposes
+any set of plugins over the feeder HTTP contract. The gateway core consumes that
+contract and keeps the libp2p wire, the bitswap blockstore, the
+hashing-into-the-blockstore and the meta-core store-back to itself.
+
+Deliberately **libp2p-free and blockstore-free**: a feeder finds and fetches
+bytes; it never joins the swarm.
+
+## Using it
+
+```toml
+[dependencies]
+meta-feeder-sdk = { git = "https://github.com/worph/meta-feeder-sdk", tag = "v0.1.0" }
+```
+
+Not on crates.io yet — the contract is still moving, so consumers pin a tag.
+
+```rust
+use meta_feeder_sdk::{serve_feeders, FeederPlugin};
+
+#[tokio::main]
+async fn main() -> anyhow::Result<()> {
+    let plugins: Vec<Box<dyn FeederPlugin>> = vec![Box::new(MyPlugin::new())];
+    serve_feeders(plugins, "/data/meta-feeder".into(), "0.0.0.0:8080".parse()?).await
+}
+```
+
+A **service is a `Vec`** — grouping plugins into one binary is a deployment
+choice, not an architectural one.
+
+## Writing a plugin
+
+Four required methods; everything else has a default:
+
+| method | you must | notes |
+|---|---|---|
+| `upstream_id` | return a stable `&'static str` | identifies your records |
+| `configure` | set up a cache dir; return `MissingConfig` if unconfigured | the harness soft-skips a plugin that returns `MissingConfig`, so an unconfigured plugin degrades instead of killing the sidecar |
+| `handle_query` | answer a `GatewayQuery` with `DiscoveryRecord`s | **open with `query_eval::query_accepts_plugin`** — see below |
+| `compute_outcomes` | resolve a `record_id` to `HashOutcome`s | `bytes: None` for metadata-only sources |
+
+Defaults you can override: `handle_query_stream`, `handle_fetch`, `health`,
+`served_file_types`, `served_content_kinds`, `config_schema`, `config_values`,
+`get_blob`.
+
+### The routing gate
+
+The gateway dispatches on `(fileType, contentKind)`, never on a service name.
+Advertise your axes and gate on them, or you will be handed queries you cannot
+answer:
+
+```rust
+fn served_file_types(&self)    -> &'static [&'static str] { &["video"] }
+fn served_content_kinds(&self) -> &'static [&'static str] { &["movie", "series"] }
+
+async fn handle_query(&self, q: &GatewayQuery, n: usize) -> Result<Vec<DiscoveryRecord>, GatewayError> {
+    if !meta_feeder_sdk::query_eval::query_accepts_plugin(
+        q, self.served_file_types(), self.served_content_kinds()) {
+        return Ok(Vec::new());
+    }
+    // ...
+}
+```
+
+`&["*"]` on an axis means "I serve every value on it".
+
+### Three rules that are not obvious
+
+1. **Records must carry a type filter.** A record with no `fileType` /
+   `contentKind` never reaches a client wall, and the symptom looks like a
+   transport bug rather than a metadata one.
+2. **`filters` / `ranges` / `negations` are authoritative; `raw_text` is
+   informational.** Consume one source per filter or you will double-filter and
+   under-match.
+3. **Never degrade an unresolvable anchor into a free-text query.**
+   `free_text_or_star()` exists for upstreams that 4xx on empty input — reaching
+   for it when an *identity* filter failed to resolve dumps the upstream's
+   recent feed into someone's library.
+
+## Hash families
+
+`HashKind` selects how the core routes an outcome. Only `Sha2_256` is seeded
+into the bitswap blockstore.
+
+| kind | codec | means |
+|---|---|---|
+| `Sha2_256` | standard | real bytes, retrievable via bitswap |
+| `Midhash256` | `0x1000` | size + middle-sample; MetaMesh-internal |
+| `BtV1File` | `0x1001` | a file inside a torrent — locator |
+| `NzbRelease` | `0x1005` | a Usenet release — locator, redeemed by a credentialed peer |
+| `CardLocator` | `0x1007` | a *work*, no bytes anywhere |
+| `YtVideo` | `0x1008` | **delegated playback** — bytes are permanently someone else's |
+
+The last two carry an identity multihash. Anything that dispatches on the
+multihash instead of the **codec** will silently mis-rank them.
+
+## Layout
+
+| module | what |
+|---|---|
+| `plugin` | the `FeederPlugin` trait, `HashKind`, `HashOutcome`, `ConfigError` |
+| `types` | `DiscoveryRecord`, `GatewayError`, `PluginHealth`, `ByteStream` |
+| `query` / `query_eval` | the structured query and its evaluator |
+| `serve` | the axum harness and the HTTP contract DTOs |
+| `config` | self-describing config schema — the dashboard renders a form from it |
+| `cache` | per-plugin redb cache |
+| `hash` | the content-addressing families above |
+| `filename_meta` / `lang` | release-tag parsing helpers |
+| `enrich`, `meta_core`, `common` | MetaMesh-internal; not part of the plugin contract |
+
+## Development
+
+```bash
+cargo build && cargo test
+```
+
+To work on the SDK from inside a feeder checkout without pushing a tag, add a
+gitignored `.cargo/config.toml` in the feeder:
+
+```toml
+paths = ["../meta-feeder-sdk"]
+```
+
+## Context
+
+Design rationale, the feeder roster, and the naming rules live in the meta-root:
+`docs/project-architecture/feeder-architecture.md`.
+
+## License
+
+MIT