Manielle
d177e7b710
Two reasons a book's page in meta-read showed no Gutenberg edition, both found on the live stack with Frankenstein (OL450063W, etext 84): - An edition was bound to its work only when Open Library's search lists the etext under exactly one work (id_project_gutenberg). For many classics it lists it under none: no edition of OL450063W carries etext 84. Such an etext is now bound by title and author: same main title (main_title), an author with the same surname, and the most published of those works, since Open Library holds many small "Frankenstein" works by Mary Shelley. The record carries the same claim plus anchorMethod=title, so the gateway records matchedBy/openlibrary:OL...W and meta-read grades it Matched, below an identifier binding. An etext several works list (an unmerged duplicate) still gets no binding: gutenberg_etext keeps "several" apart from "none". - A work page asks for openlibraryid:OL...W with no text, which fell through to the catalogue browse (the 32 most downloaded books), so a book outside them never reached its own page. Such a query now searches Gutendex for the work's main title and its first author's surname (work_search_text); the gateway still keeps only the records bound to the work. Tests: title binding graded Matched while an identifier binding stays Anchored; a work query searches "frankenstein shelley"; main_title, surname, the most-published choice, ties and non-matches; Etext one/several/unlisted and its cache form.
meta-feeder-book
Book feeder sidecar for MetaMesh.
One binary, book-feeder, hosts two upstreams:
| Upstream | Serves | What it is |
|---|---|---|
gutenberg |
document × book |
Public-domain editions from Project Gutenberg (via the Gutendex API), with an epub url-locator per record |
openlibrary |
card × book |
Work cards from Open Library — the book identity tier (TMDB : video :: AniList : comics :: Open Library : books) |
The two meet through openlibraryid. A Gutenberg edition carries it only when
Open Library itself records the link: exactly one Open Library work lists the
etext's Project Gutenberg id (search.json?q=id_project_gutenberg:<id>). The
edition then also carries the transient trust claim anchored=true,
anchorSource=openlibrary, which the gateway stores as
anchoredBy/openlibrary:OL…W. There is no title or author matching.
An Open Library card is addressed compute_card_cid("openlibrary", "OL…W"),
the bare work id, byte-identical to meta-read's card_cid::openlibrary. A work
with no cover or no description gets no card.
Role in MetaMesh
A feeder is a stateless HTTP sidecar. It finds records and fetches bytes; it
does not talk to meta-core or the libp2p blockstore. The gateway core that
calls it owns the meta-core store-back and the blockstore seeding. A gateway
registers this feeder as a RemoteFeederPlugin pointing at its / and then
drives the contract:
| Endpoint | Purpose |
|---|---|
GET /manifest |
feeder identity + capabilities |
GET /health |
liveness |
POST /query, POST /query_stream |
structured search against the upstream |
POST /compute |
enrichment / outcome compute |
GET /fetch/:upstream_id/:record_id |
fetch a record's bytes |
GET /blob/:upstream_id/:cid |
fetch a content-addressed blob |
GET /config, GET /config/schema, GET|PUT /config/values |
runtime config UI + API |
Configuration
| Env var | Default | Notes |
|---|---|---|
META_FEEDER_HTTP_LISTEN |
0.0.0.0:8080 |
HTTP listen address |
META_FEEDER_STATE_DIR |
/data/meta-feeder |
redb cache + state |
OPENLIBRARY_CONTACT |
— | e-mail or URL sent in the Open Library User-Agent. First-boot seed for the contact config field; a value saved in the config UI wins |
OPENLIBRARY_TOP_N |
12 |
cards per search |
OPENLIBRARY_DISCOVERY_N |
20 |
cards per browse row (popular:true → Open Library trending, fresh:true → recently added) |
RUST_LOG |
info |
tracing filter |
No upstream API key is required. Open Library asks API clients to identify
themselves, so set a contact in production: an identified client gets a larger
request budget. Both upstreams share one Open Library rate budget. The config
page edits the gutenberg upstream, and the contact saved there is read by
both.
Image
ghcr.io/worph/meta-feeder-book
Exposes 8080. Built and pushed by CI on every push to main (the main
tag) and on v* tags (semver tags).
Build locally
The build context is the repo root (the Cargo workspace).
meta-feeder-sdk is a git dependency pinned by tag, so the builder needs
network access to github.com:
docker build -f feeder-plugin/book-feeder/Dockerfile -t ghcr.io/worph/meta-feeder-book:dev .
Tests (cargo is not needed on the host; wiremock stands in for every upstream):
docker run --rm -v "$PWD":/src -w /src -e CARGO_TARGET_DIR=/target rust:1.89 \
cargo test -p book-feeder
Repo layout
Cargo.toml # workspace: members = feeder-plugin/*
feeder-plugin/book-feeder/ # this feeder's crate + Dockerfile
src/gutenberg.rs # `gutenberg` upstream
src/openlibrary.rs # `openlibrary` upstream (cards)
src/openlibrary_client.rs # Open Library client (budget, cache, binding lookup)
crates/meta-feeder-sdk/ # old vendored SDK copy — no longer built (excluded)
meta-feeder-sdk comes from worph/meta-feeder-sdk
at the tag in feeder-plugin/book-feeder/Cargo.toml. An SDK change reaches
this feeder only when it is tagged and the pin is bumped.