Import the MetaMesh app family from Worph/AppStore
23 apps (MetaCore, the library/network/client tiers and 13 MetaFeeder*) move out of the general-purpose Worph/AppStore into their own store so the MetaMesh family releases on its own cadence. - asset URLs repointed to cdn.jsdelivr.net/gh/Worph/MetaAppStore@main - store scaffolding added (category/recommend/featured lists, licence, contributing, gitattributes) - psd-source/meta-icons-v2 icon generator follows the apps MetaMCP stays in Worph/AppStore: it is third-party metatool-ai software, not part of MetaMesh.
@@ -0,0 +1,6 @@
|
||||
# FIX CRLF always when developer has not set
|
||||
# Linux/Mac: git config --global core.autocrlf input
|
||||
# Windows: git config --global core.autocrlf true
|
||||
# Auto detect text files and perform LF normalization
|
||||
* text=auto
|
||||
* eol=lf
|
||||
@@ -0,0 +1,4 @@
|
||||
.idea/
|
||||
.DS_Store
|
||||
*.zip
|
||||
.claude/settings.local.json
|
||||
@@ -0,0 +1,241 @@
|
||||
name: metacore
|
||||
|
||||
services:
|
||||
# Backend service. Reachable on the pcs network as `metacore-app`.
|
||||
# Public ingress goes through the `metacore` hash-lock proxy below — this
|
||||
# service has no public Caddy labels and no exposed host ports.
|
||||
metacore-app:
|
||||
image: ghcr.io/worph/meta-core:1.0.12
|
||||
container_name: metacore-app
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Both values must move together or swap comes back.
|
||||
#
|
||||
# Raised 768m -> 1536m (2026-08-26): on watch.nsl.sh the Redis working set
|
||||
# ran at 73-97% of 768m, i.e. one indexing burst away from an OOM kill that
|
||||
# takes the record store (and every service reading it) down with it.
|
||||
# 1536m puts steady state near 30%.
|
||||
mem_limit: 1536m
|
||||
memswap_limit: 1536m
|
||||
hostname: metacore-app
|
||||
restart: unless-stopped
|
||||
privileged: true
|
||||
cpu_shares: 50
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
volumes:
|
||||
# Shared infrastructure state (leader election, Redis dump, service discovery).
|
||||
# Lives inside /DATA/AppData/metacore/ so CasaOS archives it on uninstall.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metacore/meta-core:/meta-core:rw
|
||||
|
||||
# User's media directories — bind-mounted directly (no parent named volume,
|
||||
# so `docker volume rm` cannot recurse into these and wipe host data).
|
||||
- ${DATA_ROOT:-/DATA}/Downloads:/files/watch/downloads:ro
|
||||
- ${DATA_ROOT:-/DATA}/Media:/files/watch/media:ro
|
||||
|
||||
# Application cache storage
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metacore/cache:/data/cache
|
||||
|
||||
# Plugin output — gateway/feeder-published artwork (posters, backdrops)
|
||||
# lands here via meta-core's WebDAV endpoint, and every record's
|
||||
# `filePath` points into it. This MUST be a bind mount: without it the
|
||||
# directory is the container's writable layer, so any `docker compose
|
||||
# up -d` that recreates metacore-app silently deletes every poster while
|
||||
# Redis keeps pointing at them. That is exactly what happened on
|
||||
# watch.nsl.sh on 2026-08-03 (270 files → 0), and it is the reason a
|
||||
# fraction of poster CIDs 404 with a healthy-looking record behind them.
|
||||
# metacore-app is the only service that mounts /files; the gateway and
|
||||
# feeders write here over HTTP, so this one mount covers all of them.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metacore/plugin:/files/plugin:rw
|
||||
|
||||
environment:
|
||||
- BASE_URL=https://metacore-$APP_DOMAIN
|
||||
- TZ=$TZ
|
||||
|
||||
# Runs as root for nginx, Redis, rclone mounts
|
||||
user: 0:0
|
||||
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
start_period: 30s
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
# OIDC perimeter for the MetaCore dashboard. Self-registers as an OIDC client
|
||||
# with the platform's auth-registrar on first login (no secrets to manage).
|
||||
#
|
||||
# Naming convention (per appshield + auth-registrar): the proxy's
|
||||
# container_name MUST equal the URL-prefix of the public hostname, because the
|
||||
# registrar's redirect-URI allowlist is "<container_name>" or "<container_name>-*".
|
||||
# Our public URL is `metacore-${APP_DOMAIN}`, so the proxy container is named
|
||||
# `metacore` and the backend takes the `-app` suffix.
|
||||
metacore:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metacore
|
||||
hostname: metacore
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
BACKEND_HOST: "metacore-app"
|
||||
BACKEND_PORT: "80"
|
||||
LISTEN_PORT: "80"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
|
||||
depends_on:
|
||||
- metacore-app
|
||||
|
||||
labels:
|
||||
# Only /health bypasses hash-lock — inter-service traffic (/meta/*,
|
||||
# /webdav/*, /file/cid) flows over the pcs network from sibling apps
|
||||
# (metasort-app, metafuse-app, metastremio) directly to metacore-app
|
||||
# and never traverses the public proxy.
|
||||
|
||||
# ---- metacore-${APP_DOMAIN} (gateway TLS) ----
|
||||
caddy_0: metacore-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.0_handle: "/health"
|
||||
caddy_0.0_handle.reverse_proxy: "metacore-app:80"
|
||||
caddy_0.1_handle: ""
|
||||
caddy_0.1_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metacore-IP.nip.io (gateway TLS) ----
|
||||
caddy_1: metacore-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.0_handle: "/health"
|
||||
caddy_1.0_handle.reverse_proxy: "metacore-app:80"
|
||||
caddy_1.1_handle: ""
|
||||
caddy_1.1_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metacore-IP.sslip.io (Let's Encrypt) ----
|
||||
caddy_2: metacore-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.0_handle: "/health"
|
||||
caddy_2.0_handle.reverse_proxy: "metacore-app:80"
|
||||
caddy_2.1_handle: ""
|
||||
caddy_2.1_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**The infrastructure foundation for the MetaMesh media management ecosystem.**
|
||||
|
||||
MetaCore provides the shared services that power all MetaMesh applications: distributed leader election, Redis database, WebDAV file serving, remote mount management, and service discovery.
|
||||
|
||||
**Key Features:**
|
||||
- **Leader Election** - Automatic flock-based leader election with failover support
|
||||
- **Redis Database** - Centralized metadata storage for all MetaMesh services
|
||||
- **WebDAV File Serving** - Cached file access for all services via HTTP
|
||||
- **Mount Management** - Add NFS, SMB, and rclone remote mounts from the dashboard
|
||||
- **File Watcher** - Monitors folders and notifies services of new files
|
||||
- **Metadata Editor** - Browse and edit file metadata stored in Redis
|
||||
- **Service Discovery** - Automatic registration and discovery of MetaMesh services
|
||||
- **OIDC Single Sign-On** - Dashboard protected by Yundera's built-in Authelia (auto-registered, no secrets to configure)
|
||||
|
||||
**Install this FIRST** before MetaSort, MetaFuse, or MetaStremio.
|
||||
|
||||
**Dashboard Features:** Service health, file watchers, mount management, metadata editor, KV browser
|
||||
|
||||
fr_fr: |
|
||||
**La fondation d'infrastructure pour l'ecosysteme de gestion de medias MetaMesh.**
|
||||
|
||||
MetaCore fournit les services partages qui alimentent toutes les applications MetaMesh : election de leader distribuee, base de donnees Redis, service de fichiers WebDAV, gestion de montages distants et decouverte de services.
|
||||
|
||||
**Fonctionnalites Cles:**
|
||||
- **Election de Leader** - Election automatique basee sur flock avec support de basculement
|
||||
- **Base de Donnees Redis** - Stockage centralise des metadonnees pour tous les services
|
||||
- **Service de Fichiers WebDAV** - Acces aux fichiers en cache via HTTP
|
||||
- **Gestion des Montages** - Ajout de montages NFS, SMB et rclone depuis le tableau de bord
|
||||
- **Surveillance de Fichiers** - Surveille les dossiers et notifie les services des nouveaux fichiers
|
||||
- **Editeur de Metadonnees** - Parcourir et editer les metadonnees stockees dans Redis
|
||||
- **Decouverte de Services** - Enregistrement et decouverte automatiques des services
|
||||
- **SSO OIDC** - Tableau de bord protege par l'Authelia integre de Yundera (auto-enregistre, aucun secret a configurer)
|
||||
|
||||
**Installez ceci EN PREMIER** avant MetaSort, MetaFuse ou MetaStremio.
|
||||
|
||||
tagline:
|
||||
en_us: MetaMesh infrastructure - Leader election, Redis, WebDAV, and service discovery
|
||||
fr_fr: Infrastructure MetaMesh - Election de leader, Redis, WebDAV et decouverte de services
|
||||
|
||||
title:
|
||||
en_us: MetaCore
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/thumbnail.png
|
||||
|
||||
index: /
|
||||
store_app_id: metacore
|
||||
main: metacore
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**MetaCore is the foundation for the MetaMesh ecosystem.**
|
||||
|
||||
Install this FIRST before MetaSort, MetaFuse, or MetaStremio.
|
||||
|
||||
**What MetaCore can see:** it mounts your **`/DATA/Downloads`** and **`/DATA/Media`**
|
||||
folders **read-only** so it can watch them for new files and index what it finds.
|
||||
It cannot modify, rename or delete anything in them. Everything it writes stays in
|
||||
`/DATA/AppData/metacore/`. See `rationale.md` for the full breakdown.
|
||||
|
||||
**Authentication:** The dashboard is protected by Yundera's built-in Authelia via OIDC.
|
||||
Sign in with your existing Yundera account when you open the dashboard for the first time.
|
||||
|
||||
**After installation:**
|
||||
1. Open the dashboard at `https://metacore-$APP_DOMAIN/`
|
||||
2. Sign in via the Authelia portal that appears
|
||||
3. Install MetaSort, MetaFuse, or MetaStremio as needed
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metacore/meta-core
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metacore/cache
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metacore/plugin
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metacore/plugin/share
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metacore-${domain}
|
||||
webui-path: "/"
|
||||
|
After Width: | Height: | Size: 146 KiB |
@@ -0,0 +1,142 @@
|
||||
# =============================================================================
|
||||
# Authelia Configuration for MetaMesh (OIDC provider mode)
|
||||
#
|
||||
# Template — MetaCore/pre-install-cmd substitutes the placeholders below at
|
||||
# install time: APP_DOMAIN, APP_PUBLIC_IP_DASH, HMAC secret, RSA JWKS private
|
||||
# key, and one argon2id hash per OIDC client.
|
||||
#
|
||||
# Protected apps reach Authelia via OIDC (not forward_auth) so each app's
|
||||
# session cookie can be host-scoped to its own <app>-__AUTH_DOMAIN__ host.
|
||||
# =============================================================================
|
||||
|
||||
# Server settings (plain HTTP - Caddy edge proxy handles TLS).
|
||||
# Buffers bumped to cope with large OIDC requests (long state + cookies).
|
||||
server:
|
||||
address: 'tcp://0.0.0.0:9091'
|
||||
buffers:
|
||||
read: 16384
|
||||
write: 16384
|
||||
|
||||
# Logging
|
||||
log:
|
||||
level: info
|
||||
|
||||
# Authentication backend (file-based, single user)
|
||||
authentication_backend:
|
||||
file:
|
||||
path: /config/users_database.yml
|
||||
|
||||
# Session configuration — scoped to Authelia's own portal only.
|
||||
# Host-exact scope: cookie is set on 'auth-<domain>' and authelia_url is
|
||||
# 'https://auth-<domain>' — exact host match, valid cookie scope.
|
||||
# Protected apps get their own host-scoped sessions via oauth2-proxy sidecars.
|
||||
session:
|
||||
cookies:
|
||||
- domain: 'auth-__AUTH_DOMAIN__'
|
||||
authelia_url: 'https://auth-__AUTH_DOMAIN__'
|
||||
name: 'authelia_session'
|
||||
same_site: 'lax'
|
||||
inactivity: '5m'
|
||||
expiration: '1h'
|
||||
remember_me: '1M'
|
||||
|
||||
# Storage (SQLite for single-user deployment)
|
||||
storage:
|
||||
local:
|
||||
path: /config/db.sqlite
|
||||
|
||||
# Access control
|
||||
access_control:
|
||||
default_policy: one_factor
|
||||
|
||||
# TOTP (for 2FA)
|
||||
totp:
|
||||
issuer: 'MetaMesh'
|
||||
period: 30
|
||||
skew: 1
|
||||
|
||||
# Filesystem notifier (no SMTP needed for single-user)
|
||||
notifier:
|
||||
filesystem:
|
||||
filename: /config/notifications.txt
|
||||
|
||||
# OIDC Identity Provider — each MetaMesh app registers as a client and
|
||||
# delegates auth to Authelia via the standard Authorization Code flow.
|
||||
identity_providers:
|
||||
oidc:
|
||||
hmac_secret: '__OIDC_HMAC__'
|
||||
jwks:
|
||||
- key_id: 'metamesh-default'
|
||||
algorithm: 'RS256'
|
||||
use: 'sig'
|
||||
key: |
|
||||
__OIDC_JWKS_KEY__
|
||||
clients:
|
||||
- client_id: 'metacore'
|
||||
client_name: 'MetaCore Dashboard'
|
||||
client_secret: '__CLIENT_METACORE_HASH__'
|
||||
public: false
|
||||
authorization_policy: 'one_factor'
|
||||
consent_mode: 'implicit'
|
||||
require_pkce: false
|
||||
redirect_uris:
|
||||
- 'https://metacore-__AUTH_DOMAIN__/oauth2/callback'
|
||||
- 'https://metacore-__APP_PUBLIC_IP_DASH__.nip.io/oauth2/callback'
|
||||
- 'https://metacore-__APP_PUBLIC_IP_DASH__.sslip.io/oauth2/callback'
|
||||
scopes:
|
||||
- 'openid'
|
||||
- 'profile'
|
||||
- 'email'
|
||||
- 'groups'
|
||||
response_types:
|
||||
- 'code'
|
||||
grant_types:
|
||||
- 'authorization_code'
|
||||
- 'refresh_token'
|
||||
token_endpoint_auth_method: 'client_secret_basic'
|
||||
|
||||
- client_id: 'metasort'
|
||||
client_name: 'MetaSort Dashboard'
|
||||
client_secret: '__CLIENT_METASORT_HASH__'
|
||||
public: false
|
||||
authorization_policy: 'one_factor'
|
||||
consent_mode: 'implicit'
|
||||
require_pkce: false
|
||||
redirect_uris:
|
||||
- 'https://metasort-__AUTH_DOMAIN__/oauth2/callback'
|
||||
- 'https://metasort-__APP_PUBLIC_IP_DASH__.nip.io/oauth2/callback'
|
||||
- 'https://metasort-__APP_PUBLIC_IP_DASH__.sslip.io/oauth2/callback'
|
||||
scopes:
|
||||
- 'openid'
|
||||
- 'profile'
|
||||
- 'email'
|
||||
- 'groups'
|
||||
response_types:
|
||||
- 'code'
|
||||
grant_types:
|
||||
- 'authorization_code'
|
||||
- 'refresh_token'
|
||||
token_endpoint_auth_method: 'client_secret_basic'
|
||||
|
||||
- client_id: 'metafuse'
|
||||
client_name: 'MetaFuse Dashboard'
|
||||
client_secret: '__CLIENT_METAFUSE_HASH__'
|
||||
public: false
|
||||
authorization_policy: 'one_factor'
|
||||
consent_mode: 'implicit'
|
||||
require_pkce: false
|
||||
redirect_uris:
|
||||
- 'https://metafuse-__AUTH_DOMAIN__/oauth2/callback'
|
||||
- 'https://metafuse-__APP_PUBLIC_IP_DASH__.nip.io/oauth2/callback'
|
||||
- 'https://metafuse-__APP_PUBLIC_IP_DASH__.sslip.io/oauth2/callback'
|
||||
scopes:
|
||||
- 'openid'
|
||||
- 'profile'
|
||||
- 'email'
|
||||
- 'groups'
|
||||
response_types:
|
||||
- 'code'
|
||||
grant_types:
|
||||
- 'authorization_code'
|
||||
- 'refresh_token'
|
||||
token_endpoint_auth_method: 'client_secret_basic'
|
||||
@@ -0,0 +1,10 @@
|
||||
# Authelia Users Database for MetaMesh
|
||||
# Generated by pre-install-cmd with hashed password
|
||||
users:
|
||||
admin:
|
||||
disabled: false
|
||||
displayname: 'Admin'
|
||||
password: '__PASSWORD_HASH__'
|
||||
email: '[email protected]'
|
||||
groups:
|
||||
- admins
|
||||
@@ -0,0 +1,54 @@
|
||||
# MetaCore — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metacore-app` runs `privileged: true` and `user: 0:0`.
|
||||
2. It mounts two broad slices of the user's data area, read-only:
|
||||
`/DATA/Downloads` and `/DATA/Media`.
|
||||
3. `/health` bypasses the SSO gate and is proxied straight to the backend.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. MetaCore is the MetaMesh infrastructure container: it runs nginx, Redis and — the
|
||||
load-bearing part — **rclone remote mounts** (NFS, SMB, S3, WebDAV) that the user adds
|
||||
from the dashboard. Creating a mount namespace inside the container needs `SYS_ADMIN`
|
||||
and `/dev/fuse`; the image also runs several supervised daemons that bind their own
|
||||
sockets. Neither works from an unprivileged, uid-1000 process.
|
||||
2. MetaCore is the file watcher for the whole ecosystem. It has to see the directories a
|
||||
user actually drops media into, which are exactly `/DATA/Downloads` and `/DATA/Media`.
|
||||
There is no narrower path that satisfies the feature.
|
||||
3. Sibling MetaMesh apps and the PCS health probe need an unauthenticated liveness
|
||||
endpoint; routing it to the backend keeps the tile's health independent of whether
|
||||
Authelia is up.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- **Both user-data mounts are `:ro`.** MetaCore reads and indexes; it cannot write to,
|
||||
rename or delete anything under `/DATA/Downloads` or `/DATA/Media`.
|
||||
- Everything MetaCore writes lives under `/DATA/AppData/metacore/`, declared in
|
||||
`x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- The dashboard itself is behind AppShield + Authelia SSO; only `/health` is exempt, and
|
||||
it returns a liveness object with no user data in it.
|
||||
- `mem_limit`/`memswap_limit` are both 768 MB, so the container is OOM-killed rather than
|
||||
driving the host into swap.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Drop `privileged` for a `cap_add` list.** Tried: rclone's FUSE mounts need
|
||||
`SYS_ADMIN` plus an unconfined AppArmor profile plus `/dev/fuse`, which is most of what
|
||||
`privileged` grants, and the remaining supervised daemons still failed to start. The
|
||||
reduced form was not meaningfully smaller, and it was harder to reason about.
|
||||
- **Run as `$PUID:$PGID`.** The mount syscalls fail outright.
|
||||
- **Mount narrower paths than `/DATA/Downloads` / `/DATA/Media`.** These *are* the user's
|
||||
download and media roots; a subdirectory would silently miss most of their library.
|
||||
- **Put `/health` behind the gate.** The PCS probe has no Authelia session, so the tile
|
||||
would show unhealthy whenever SSO was restarting.
|
||||
|
||||
## Data protection
|
||||
|
||||
User media is mounted read-only and never modified. All MetaCore state (Redis dump,
|
||||
leader-election files, plugin artwork, cache) is under `/DATA/AppData/metacore/`, which is
|
||||
the unit Maison archives on uninstall and restores from backup — so an uninstall/reinstall
|
||||
cycle with "keep user data" comes back with metadata, posters and mounts intact. The
|
||||
`plugin/` directory is a bind mount specifically so that recreating the container cannot
|
||||
take published artwork with it.
|
||||
|
After Width: | Height: | Size: 76 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 143 KiB |
|
After Width: | Height: | Size: 146 KiB |
@@ -0,0 +1,168 @@
|
||||
name: metafeederanilist
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · AniList — the comics/manga IDENTITY tier for MetaMesh.
|
||||
#
|
||||
# AniList names the *work*: title, cover, synopsis, author, reading direction,
|
||||
# and the `anilistid` chapters anchor on. Keyless, cheap, no config. It serves no
|
||||
# chapters and must not. Split out of the former MetaFeeder · Comic app; pairs
|
||||
# with **MetaFeeder · Suwayomi** (the chapters).
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeederanilist-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeederanilist-*` prefix + the per-stack `metafeederanilist` network +
|
||||
# /DATA/AppData/metafeederanilist/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederanilist-feeder — the identity tier. Answers "this series exists; here
|
||||
# are its ids, cover, synopsis, reading direction, author." Keyless.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederanilist-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-anilist:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeederanilist-feeder
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeederanilist-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info,anilist_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederanilist/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeederanilist
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederanilist:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederanilist
|
||||
hostname: metafeederanilist
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederanilist-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederanilist-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederanilist-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederanilist-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederanilist-feeder
|
||||
networks:
|
||||
- metafeederanilist
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederanilist:
|
||||
name: metafeederanilist
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederanilist
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**The comics and manga identity tier for MetaMesh.**
|
||||
|
||||
MetaFeeder · AniList names the *work* — title, cover, synopsis, author, reading
|
||||
direction, and the id chapters anchor on. Keyless and cheap.
|
||||
|
||||
**Pairs with MetaFeeder · Suwayomi**, which supplies the chapters: chapters
|
||||
matched to an AniList work converge on **one** work page instead of
|
||||
fragmenting into one tile per romanisation. **Pairs with MetaRead**, the reading
|
||||
client.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**La couche d'identité BD et manga de MetaMesh.**
|
||||
|
||||
MetaFeeder · AniList nomme l'*œuvre* — titre, couverture, synopsis, auteur, sens
|
||||
de lecture, et l'identifiant sur lequel les chapitres s'ancrent. Sans clé.
|
||||
|
||||
**Va avec MetaFeeder · Suwayomi** (les chapitres) et **MetaRead**. **Nécessite**
|
||||
l'app **MetaGateway**.
|
||||
tagline:
|
||||
en_us: AniList — the comics and manga identity tier for MetaMesh
|
||||
fr_fr: AniList — la couche d'identité BD et manga de MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · AniList
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederAniList/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederAniList/thumbnail.png
|
||||
|
||||
store_app_id: metafeederanilist
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores and
|
||||
announces what this feeder finds).
|
||||
|
||||
**After installing this app:** in **MetaGateway**, add this feeder from the
|
||||
dashboard's *Add a feeder* box — name `anilist-feeder`, URL `http://metafeederanilist-feeder:8080`.
|
||||
The gateway restarts itself: feeder discovery is startup-only.
|
||||
No other setup. For chapters, also install **MetaFeeder · Suwayomi**.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederanilist/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederanilist-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 132 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · AniList — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeederanilist-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Comic), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeederanilist/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeederanilist/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 132 KiB |
@@ -0,0 +1,156 @@
|
||||
name: metafeederarxiv
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · arXiv — preprints for MetaMesh.
|
||||
#
|
||||
# Keyless: finds papers on arXiv and returns the records to the MetaGateway
|
||||
# core, which stores + announces them. Split out of the former
|
||||
# MetaFeeder · Papers app (one provider per app).
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeederarxiv-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeederarxiv-*` prefix + the per-stack `metafeederarxiv` network +
|
||||
# /DATA/AppData/metafeederarxiv/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederarxiv-feeder — the arxiv feeder. Keyless; no config needed.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederarxiv-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-arxiv:1.0.1
|
||||
cpu_shares: 30
|
||||
container_name: metafeederarxiv-feeder
|
||||
hostname: metafeederarxiv-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederarxiv/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeederarxiv
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederarxiv:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederarxiv
|
||||
hostname: metafeederarxiv
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederarxiv-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederarxiv-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederarxiv-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederarxiv-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederarxiv-feeder
|
||||
networks:
|
||||
- metafeederarxiv
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederarxiv:
|
||||
name: metafeederarxiv
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederarxiv
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**arXiv for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · arXiv bridges open-access preprints from **arXiv** into MetaMesh.
|
||||
Keyless — no setup required.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**arXiv pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · arXiv connecte les prépublications en libre accès d'**arXiv** à
|
||||
MetaMesh. Sans clé ni configuration. **Nécessite** l'app **MetaGateway**.
|
||||
tagline:
|
||||
en_us: arXiv open-access preprints for MetaMesh
|
||||
fr_fr: Prépublications arXiv en libre accès pour MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · arXiv
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederArxiv/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederArxiv/thumbnail.png
|
||||
|
||||
store_app_id: metafeederarxiv
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores and
|
||||
announces what this feeder finds).
|
||||
|
||||
**After installing this app:** in **MetaGateway**, add this feeder from the
|
||||
dashboard's *Add a feeder* box — name `arxiv-feeder`, URL `http://metafeederarxiv-feeder:8080`.
|
||||
The gateway restarts itself: feeder discovery is startup-only.
|
||||
No other setup.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederarxiv/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederarxiv-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 131 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · arXiv — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeederarxiv-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Papers), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeederarxiv/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeederarxiv/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 131 KiB |
@@ -0,0 +1,157 @@
|
||||
name: metafeedereuropepmc
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Europe PMC — biomedical literature (PubMed) for MetaMesh.
|
||||
#
|
||||
# Keyless: searches Europe PMC (PubMed and PubMed Central) and returns the
|
||||
# records to the MetaGateway core, which stores + announces them. The upstream
|
||||
# id is `pubmed`. Split out of the former MetaFeeder · Papers app.
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeedereuropepmc-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeedereuropepmc-*` prefix + the per-stack `metafeedereuropepmc` network +
|
||||
# /DATA/AppData/metafeedereuropepmc/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeedereuropepmc-feeder — the europepmc feeder (upstream `pubmed`). Keyless.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedereuropepmc-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-europepmc:1.0.1
|
||||
cpu_shares: 30
|
||||
container_name: metafeedereuropepmc-feeder
|
||||
hostname: metafeedereuropepmc-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedereuropepmc/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeedereuropepmc
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedereuropepmc:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeedereuropepmc
|
||||
hostname: metafeedereuropepmc
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeedereuropepmc-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeedereuropepmc-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeedereuropepmc-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeedereuropepmc-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeedereuropepmc-feeder
|
||||
networks:
|
||||
- metafeedereuropepmc
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeedereuropepmc:
|
||||
name: metafeedereuropepmc
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeedereuropepmc
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Europe PMC (PubMed) for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Europe PMC bridges biomedical literature from **Europe PMC** —
|
||||
PubMed and PubMed Central — into MetaMesh. Keyless — no setup required.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**Europe PMC (PubMed) pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Europe PMC connecte la littérature biomédicale d'**Europe PMC**
|
||||
— PubMed et PubMed Central — à MetaMesh. Sans clé ni configuration.
|
||||
**Nécessite** l'app **MetaGateway**.
|
||||
tagline:
|
||||
en_us: Europe PMC / PubMed biomedical literature for MetaMesh
|
||||
fr_fr: Littérature biomédicale Europe PMC / PubMed pour MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Europe PMC
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederEuropePMC/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederEuropePMC/thumbnail.png
|
||||
|
||||
store_app_id: metafeedereuropepmc
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores and
|
||||
announces what this feeder finds).
|
||||
|
||||
**After installing this app:** in **MetaGateway**, add this feeder from the
|
||||
dashboard's *Add a feeder* box — name `europepmc-feeder`, URL `http://metafeedereuropepmc-feeder:8080`.
|
||||
The gateway restarts itself: feeder discovery is startup-only.
|
||||
No other setup.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeedereuropepmc/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeedereuropepmc-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 131 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · Europe PMC — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeedereuropepmc-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Papers), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeedereuropepmc/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeedereuropepmc/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 131 KiB |
@@ -0,0 +1,161 @@
|
||||
name: metafeedergiphy
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Giphy — GIFs for MetaMesh.
|
||||
#
|
||||
# Opt-in: soft-skips (healthy, answers nothing) unless GIPHY_API_KEY is set.
|
||||
# Split out of the former MetaFeeder · Commons app (one provider per app).
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeedergiphy-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeedergiphy-*` prefix + the per-stack `metafeedergiphy` network +
|
||||
# /DATA/AppData/metafeedergiphy/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeedergiphy-feeder — the giphy feeder. Needs a Giphy API key.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedergiphy-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-giphy:1.0.0
|
||||
cpu_shares: 30
|
||||
container_name: metafeedergiphy-feeder
|
||||
hostname: metafeedergiphy-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
# Giphy is opt-in. Blank → the feeder soft-skips. This image has no config-UI
|
||||
# field for the key yet, so it is read from the environment.
|
||||
GIPHY_API_KEY: "${GIPHY_API_KEY:-}"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedergiphy/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeedergiphy
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedergiphy:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeedergiphy
|
||||
hostname: metafeedergiphy
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeedergiphy-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeedergiphy-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeedergiphy-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeedergiphy-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeedergiphy-feeder
|
||||
networks:
|
||||
- metafeedergiphy
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeedergiphy:
|
||||
name: metafeedergiphy
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeedergiphy
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Giphy for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Giphy bridges **Giphy** GIFs into MetaMesh. It needs a Giphy API
|
||||
key; without one it stays healthy and simply finds nothing.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**Giphy pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Giphy connecte les GIFs **Giphy** à MetaMesh. Nécessite une clé
|
||||
d'API Giphy ; sans elle, il reste sain et ne trouve rien. **Nécessite** l'app
|
||||
**MetaGateway**.
|
||||
tagline:
|
||||
en_us: Giphy GIFs for MetaMesh (API key required)
|
||||
fr_fr: GIFs Giphy pour MetaMesh (clé d'API requise)
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Giphy
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederGiphy/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederGiphy/thumbnail.png
|
||||
|
||||
store_app_id: metafeedergiphy
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores and
|
||||
announces what this feeder finds).
|
||||
|
||||
**After installing this app:** in **MetaGateway**, add this feeder from the
|
||||
dashboard's *Add a feeder* box — name `giphy-feeder`, URL `http://metafeedergiphy-feeder:8080`.
|
||||
The gateway restarts itself: feeder discovery is startup-only.
|
||||
|
||||
**API key:** set `GIPHY_API_KEY` in this app's environment (developers.giphy.com),
|
||||
then restart the app. Without it the feeder soft-skips.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeedergiphy/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeedergiphy-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · Giphy — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeedergiphy-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Commons), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeedergiphy/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeedergiphy/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,155 @@
|
||||
name: metafeedergutenberg
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Gutenberg — a standalone Project Gutenberg feeder for MetaMesh.
|
||||
#
|
||||
# Split out of MetaGateway so you can run only the feeders you want. Headless
|
||||
# (no enrichment, no config needed): it finds public-domain books and returns
|
||||
# the records to the MetaGateway core, which stores + announces them. The
|
||||
# gateway must list this feeder's URL (`http://metafeedergutenberg-feeder:8080`) in
|
||||
# its feeders config (both on the shared `pcs` network).
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeedergutenberg-feeder — the gutenberg feeder. The gateway reaches it on `pcs`
|
||||
# for discovery/query; it returns records (no meta-core, no enrichment).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedergutenberg-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-gutenberg:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeedergutenberg-feeder
|
||||
hostname: metafeedergutenberg-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedergutenberg/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeedergutenberg
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedergutenberg:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeedergutenberg
|
||||
hostname: metafeedergutenberg
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeedergutenberg-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeedergutenberg-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeedergutenberg-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeedergutenberg-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeedergutenberg-feeder
|
||||
networks:
|
||||
- metafeedergutenberg
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeedergutenberg:
|
||||
name: metafeedergutenberg
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeedergutenberg
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**A standalone Project Gutenberg feeder for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Gutenberg bridges public-domain books from Project Gutenberg into
|
||||
MetaMesh so they surface across the federation, plus **Open Library** work
|
||||
cards — the book identity tier that groups editions under one work. Split out of MetaGateway so
|
||||
you can run only the feeders you want — headless, no setup required.
|
||||
|
||||
**Requires (from the store):**
|
||||
- **MetaGateway** — its meta-core stores the resolved records and announces
|
||||
them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**Un feeder Project Gutenberg autonome pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Gutenberg connecte les livres du domaine public de Project
|
||||
Gutenberg à MetaMesh. Détaché de MetaGateway pour n'exécuter que les feeders
|
||||
souhaités — sans configuration. **Nécessite** l'app **MetaGateway**.
|
||||
|
||||
tagline:
|
||||
en_us: Project Gutenberg editions + Open Library work cards for MetaMesh
|
||||
fr_fr: Éditions Project Gutenberg + fiches Open Library pour MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Gutenberg
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederGutenberg/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederGutenberg/thumbnail.png
|
||||
|
||||
store_app_id: metafeedergutenberg
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first.
|
||||
|
||||
**After installing this app:** in **MetaGateway**, make sure this feeder
|
||||
is listed in its feeders config (`http://metafeedergutenberg-feeder:8080`) so
|
||||
it gets discovered (or add it from the gateway dashboard). No other setup.
|
||||
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeedergutenberg/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeedergutenberg-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 131 KiB |
@@ -0,0 +1,41 @@
|
||||
# MetaFeeder · Gutenberg — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeedergutenberg-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
The feeder publishes its resolved records to the shared MetaCore over HTTP and writes its
|
||||
own service registration and plugin artwork through MetaCore's WebDAV endpoint. Those
|
||||
writes land in directories created and owned by MetaCore's root-running container; a
|
||||
uid-1000 process is refused. The feeder image also runs a small supervised process group
|
||||
(fetcher + enrichment plugin clients) that expects to be able to signal its children.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeedergutenberg/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Writes into MetaCore's root-owned `/meta-core` tree and through
|
||||
its WebDAV endpoint fail, and the feeder starts but silently publishes nothing.
|
||||
- **Have MetaCore relax the ownership of its shared directory.** That would loosen
|
||||
permissions for every MetaMesh app at once, which is strictly worse than one root
|
||||
container with no user-data mounts.
|
||||
- **Drop privileges after startup inside the image.** Upstream change; tracked, not
|
||||
available in the shipped image.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state — configuration, cursor position, and
|
||||
(where applicable) the TMDB cache — lives under `/DATA/AppData/metafeedergutenberg/`, which is the unit
|
||||
Maison archives on uninstall and restores from backup, so an uninstall/reinstall with "keep
|
||||
user data" resumes where it left off rather than re-indexing from scratch.
|
||||
|
After Width: | Height: | Size: 131 KiB |
@@ -0,0 +1,246 @@
|
||||
name: metafeederopensubtitles
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · OpenSubtitles — subtitle SEARCH for MetaMesh, from OpenSubtitles.
|
||||
#
|
||||
# Answers `fileType:subtitle` queries, two ways (the client picks):
|
||||
# keyword `matrix fileType:subtitle`
|
||||
# anchored `imdbid:tt0133093` · `tmdbid:1399 season:1 episode:1` ·
|
||||
# `moviehash:<16 hex>` — each optionally narrowed by `languages:fre`
|
||||
# Each result is ONE subtitle file — language, release name, download count,
|
||||
# hearing-impaired / machine-translated / exact-hash flags — addressed by a
|
||||
# `provider-file` locator ("opensubtitles", "file:<file_id>").
|
||||
#
|
||||
# Search only: it never calls OpenSubtitles' /download, so it spends none of the
|
||||
# account's download quota and needs only an API key, no login. Fetching the
|
||||
# subtitle FILES through the mesh is a later MetaShare tier — not available yet.
|
||||
#
|
||||
# Self-contained: calls OpenSubtitles directly (rate-limited, 24 h search cache),
|
||||
# holds no bytes, runs no enrichment sidecars. It does NOT talk to meta-core —
|
||||
# the MetaGateway core stores and announces the records it returns.
|
||||
#
|
||||
# Not the same thing as the `metamesh-plugin-opensubtitle` meta-sort plugin,
|
||||
# which attaches subtitles to files you already have. This app answers searches.
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeederopensubtitles-feeder:8080`) in its feeders config so it gets
|
||||
# discovered. Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeederopensubtitles-*` prefix + the per-stack
|
||||
# `metafeederopensubtitles` network + /DATA/AppData/metafeederopensubtitles/ data root, so
|
||||
# it installs alongside the other Meta* apps with zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederopensubtitles-feeder — the feeder (meta-feeder-opensubtitles) hosting
|
||||
# the `opensubtitles` upstream. The gateway core reaches it directly on `pcs`
|
||||
# for discovery/query; the human configures it through the AppShield gate below.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederopensubtitles-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-opensubtitles:1.0.0
|
||||
cpu_shares: 30
|
||||
container_name: metafeederopensubtitles-feeder
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM instead of
|
||||
# swap. One HTTP client + a redb search cache; generous, not tight.
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeederopensubtitles-feeder
|
||||
restart: unless-stopped
|
||||
# Unlike the other feeders this one never writes into MetaCore's root-owned
|
||||
# tree — its only write is its own state dir below, which Maison creates as
|
||||
# $PUID:$PGID (x-compose-app.folders). So no root is needed.
|
||||
user: $PUID:$PGID
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# INFRA-ONLY env (the "config: no env if in webui" rule). The OpenSubtitles
|
||||
# API key and User-Agent are read from the persisted config.json written by
|
||||
# this feeder's config page (behind the AppShield gate below).
|
||||
# OPENSUBTITLES_API_KEY / OPENSUBTITLES_USER_AGENT exist only as first-boot
|
||||
# seeds; don't set them here. META_FEEDER_STATE_DIR is left at its default
|
||||
# (/data/meta-feeder), which is what the volume below mounts.
|
||||
RUST_LOG: "info"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config (config.json saved via the config UI) + the redb
|
||||
# search cache. Wiping the cache is safe; it only costs upstream calls.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederopensubtitles/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeederopensubtitles
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config UI (set the OpenSubtitles API key)
|
||||
# behind Yundera OIDC SSO. container_name == subdomain prefix. The gateway
|
||||
# still reaches the feeder directly on `pcs` (this gate is for the human
|
||||
# only); index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederopensubtitles:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederopensubtitles
|
||||
hostname: metafeederopensubtitles
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederopensubtitles-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederopensubtitles-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederopensubtitles-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederopensubtitles-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederopensubtitles-feeder
|
||||
networks:
|
||||
- metafeederopensubtitles
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederopensubtitles:
|
||||
name: metafeederopensubtitles
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederopensubtitles
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Subtitle search for MetaMesh, from OpenSubtitles.**
|
||||
|
||||
MetaFeeder · OpenSubtitles answers subtitle searches across the mesh — by
|
||||
**keyword** (*"matrix"*) or **anchored on a work**: an IMDb or TMDB id,
|
||||
a season and episode, or the exact file you are playing (its OpenSubtitles
|
||||
*moviehash*). Every search can be narrowed to your languages.
|
||||
|
||||
Each result is **one subtitle file** with its language, release name,
|
||||
download count and flags — hearing-impaired, machine-translated, and
|
||||
whether it was timed against your exact file. Results come back best-first:
|
||||
exact file matches, then the most downloaded.
|
||||
|
||||
**Search only.** It never downloads, so it spends none of your
|
||||
OpenSubtitles download quota and needs only a free API key — no account
|
||||
login. Fetching the subtitle files through the mesh is a later MetaShare
|
||||
feature and is **not available yet**.
|
||||
|
||||
**Cheap and self-contained:** calls OpenSubtitles directly (rate-limited,
|
||||
searches cached for 24 h), holds no bytes, runs no enrichment sidecars.
|
||||
~256 MB RAM.
|
||||
|
||||
**Requires:** **MetaGateway** — its meta-core stores the results and
|
||||
announces them to the swarm. The gateway must list this feeder in its
|
||||
config. You also need a free **OpenSubtitles API key**.
|
||||
|
||||
**Pairs with:** **MetaWatch**.
|
||||
|
||||
fr_fr: |
|
||||
**Recherche de sous-titres pour MetaMesh, depuis OpenSubtitles.**
|
||||
|
||||
MetaFeeder · OpenSubtitles répond aux recherches de sous-titres sur le mesh —
|
||||
par **mot-clé** (*« matrix »*) ou **ancrée sur une œuvre** : un identifiant
|
||||
IMDb ou TMDB, une saison et un épisode, ou le fichier exact que vous lisez
|
||||
(son *moviehash* OpenSubtitles). Chaque recherche peut être restreinte à vos
|
||||
langues.
|
||||
|
||||
Chaque résultat est **un fichier de sous-titres** avec sa langue, le nom de
|
||||
la release, le nombre de téléchargements et des indicateurs —
|
||||
malentendants, traduction automatique, et synchronisation sur votre fichier
|
||||
exact. Les meilleurs résultats d'abord : correspondances exactes, puis les
|
||||
plus téléchargés.
|
||||
|
||||
**Recherche uniquement.** Aucun téléchargement : aucun quota OpenSubtitles
|
||||
consommé, seule une clé d'API gratuite est nécessaire — pas de connexion au
|
||||
compte. La récupération des fichiers via le mesh est une fonctionnalité
|
||||
MetaShare à venir et **n'est pas encore disponible**.
|
||||
|
||||
**Léger et autonome :** appelle OpenSubtitles directement (débit limité,
|
||||
recherches en cache 24 h), ne stocke aucun octet, aucun plugin
|
||||
d'enrichissement. ~256 Mo de RAM.
|
||||
|
||||
**Nécessite :** l'app **MetaGateway** (son meta-core stocke et annonce les
|
||||
résultats) et une **clé d'API OpenSubtitles** gratuite.
|
||||
|
||||
tagline:
|
||||
en_us: OpenSubtitles search — find subtitles by keyword, work or exact file
|
||||
fr_fr: Recherche OpenSubtitles — sous-titres par mot-clé, œuvre ou fichier exact
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · OpenSubtitles
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederOpenSubtitles/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederOpenSubtitles/thumbnail.png
|
||||
|
||||
store_app_id: metafeederopensubtitles
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores
|
||||
and announces the subtitle records this feeder finds).
|
||||
|
||||
**After installing this app:**
|
||||
1. Get a free **OpenSubtitles API key** — opensubtitles.com → sign in →
|
||||
*API consumers* → create a consumer and copy its key. No account
|
||||
password is needed here: this feeder only searches.
|
||||
2. Open this app's config UI (the dashboard link), paste it into
|
||||
**OpenSubtitles API key** and save. Leave **User-Agent** blank unless
|
||||
OpenSubtitles asks you to use your registered app name. The feeder
|
||||
restarts itself to apply the key.
|
||||
3. In **MetaGateway**, add this feeder to its feeders config —
|
||||
name `opensubtitles-feeder`, URL `http://metafeederopensubtitles-feeder:8080`
|
||||
— from the gateway dashboard's *Add a feeder* box. The gateway
|
||||
restarts itself and picks up the `opensubtitles` upstream (tag
|
||||
SUBTITLE).
|
||||
|
||||
**Without an API key** the feeder stays up but reports degraded, and
|
||||
subtitle searches simply return nothing. That is by design, not a failure.
|
||||
|
||||
**Searches must ask for subtitles** (`fileType:subtitle`). A plain search
|
||||
box query never reaches this feeder.
|
||||
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# The feeder runs as $PUID:$PGID, so this ownership is what lets it write.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederopensubtitles/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederopensubtitles-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 133 KiB |
|
After Width: | Height: | Size: 133 KiB |
@@ -0,0 +1,293 @@
|
||||
name: metafeederprowlarr
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Prowlarr — a STANDALONE Prowlarr feeder for MetaMesh.
|
||||
#
|
||||
# Bridges your **Prowlarr** indexers — Torznab/Newznab, torrent AND usenet — into
|
||||
# MetaMesh, so releases surface in **MetaWatch**. Prowlarr is the separate store
|
||||
# app (container `prowlarr:9696`) that aggregates the indexers.
|
||||
#
|
||||
# (The Tribler source used to ride in this same app; it was extracted into its
|
||||
# own image + app — install **MetaFeeder · Tribler** for decentralized IPv8
|
||||
# discovery. This app is now Prowlarr/Torznab-only.)
|
||||
#
|
||||
# Split out of the MetaGateway stack so the sources stay modular. Self-contained
|
||||
# for enrichment: ships its own filename-parser + tmdb plugins.
|
||||
#
|
||||
# It needs, all reached over the shared `pcs` network:
|
||||
# 1. the **Prowlarr** store app (`prowlarr:9696`) — the indexer source.
|
||||
# 2. the shared **MetaCore** app (`metacore-app:9000`) — where resolved
|
||||
# records + TMDB posters are stored; the shared **MetaShare** app then
|
||||
# announces them to the swarm.
|
||||
# The **MetaGateway** app must list this feeder's URL
|
||||
# (`http://metafeederprowlarr-feeder:8080`) in its feeders config so it
|
||||
# discovers + queries it (MetaGateway, MetaCore and this feeder share `pcs`).
|
||||
#
|
||||
# Everything is scoped under the `metafeederprowlarr-*` prefix + the
|
||||
# per-stack `metafeederprowlarr` network + /DATA/AppData/metafeederprowlarr/
|
||||
# data root, so it installs alongside the other Meta* apps with zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederprowlarr-feeder — the feeder (meta-feeder-torznab) running the
|
||||
# Prowlarr/Torznab source. The gateway reaches it directly on `pcs` for
|
||||
# discovery/query/compute; the human configures it through the AppShield gate
|
||||
# below.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederprowlarr-feeder:
|
||||
# Image name is `meta-feeder-torznab` — the repo (and therefore the ghcr
|
||||
# package, published as ${{ github.repository }}) was renamed. The old
|
||||
# `meta-feeder-indexer` package still exists but is frozen at 1.0.23 and
|
||||
# receives no further pushes. The APP keeps the `metafeederprowlarr` name.
|
||||
image: ghcr.io/worph/meta-feeder-torznab:1.0.26
|
||||
cpu_shares: 30
|
||||
container_name: metafeederprowlarr-feeder
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 512m
|
||||
memswap_limit: 512m
|
||||
hostname: metafeederprowlarr-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# INFRA-ONLY env (the "config: no env if in webui" rule). Operator config —
|
||||
# Prowlarr indexers + API key, TMDB token / language, meta-core URL — is
|
||||
# read from the persisted config.json (this feeder's config page behind the
|
||||
# AppShield gate); env is only a deprecated first-boot seed. These env vars
|
||||
# are wiring only: the enrichment sidecars, the enrich callback, the peer id
|
||||
# and Prowlarr paging. meta-core defaults to the pcs
|
||||
# `http://metacore-app:9000`; Prowlarr is reached as `prowlarr:9696`.
|
||||
RUST_LOG: "info,meta_feeder=info,meta_feeder::store=debug"
|
||||
# The peer id stamped into provenance.gateway_peer on every record this
|
||||
# feeder self-publishes. It names the GATEWAY this feeder serves — not the
|
||||
# feeder itself — so records are attributed to the gateway that fronts them
|
||||
# (analytics group by it). Without it, it falls back to HOSTNAME.
|
||||
META_GATEWAY_PEER_ID: "metagateway"
|
||||
FILENAME_PARSER_URL: "http://metafeederprowlarr-filename-parser:8080"
|
||||
TMDB_PLUGIN_URL: "http://metafeederprowlarr-tmdb:8080"
|
||||
META_FEEDER_CALLBACK_URL: "http://metafeederprowlarr-feeder:8080/enrich/callback"
|
||||
# LOAD-BEARING at meta-feeder-indexer:1.0.9 — NOT a restatement of the
|
||||
# default: that build defaults the page size to 100. 20 is the pacing
|
||||
# choice (results stream 20-at-a-time). META_GATEWAY_PROWLARR_MAX_DEPTH
|
||||
# was dropped — its 500 did match the default.
|
||||
META_GATEWAY_PROWLARR_PAGE_SIZE: "20"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config (config.json saved via the config UI) + redb cache.
|
||||
# Transient librqbit downloads go to the container's /tmp (off backup).
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederprowlarr/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
depends_on:
|
||||
# service_started (not _healthy): the plugin images ship no healthcheck.
|
||||
metafeederprowlarr-filename-parser:
|
||||
condition: service_started
|
||||
metafeederprowlarr-tmdb:
|
||||
condition: service_started
|
||||
networks:
|
||||
- metafeederprowlarr
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bundled enrichment plugins (self-contained). filename-parser per file; tmdb
|
||||
# once per torrent — tmdb writes posters/backdrops to the gateway core's WebDAV.
|
||||
# No healthcheck: these plugin images don't ship one (curl absent).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederprowlarr-filename-parser:
|
||||
image: ghcr.io/worph/metamesh-plugin-filename-parser:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeederprowlarr-filename-parser
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeederprowlarr-filename-parser
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
EMIT_TITLE_TAGS: "true"
|
||||
networks:
|
||||
- metafeederprowlarr
|
||||
|
||||
metafeederprowlarr-tmdb:
|
||||
image: ghcr.io/worph/metamesh-plugin-tmdb:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeederprowlarr-tmdb
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 384m
|
||||
memswap_limit: 384m
|
||||
hostname: metafeederprowlarr-tmdb
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# Posters land in the gateway core (over pcs) so the ingester seeds them.
|
||||
# LOAD-BEARING at metamesh-plugin-tmdb:1.0.0 — do NOT remove.
|
||||
# Newer source derives WebDAV from the metaCoreUrl passed on each /process
|
||||
# (GET /urls -> webdavUrlInternal), which is why the dev compose omits it. The
|
||||
# PUBLISHED image predates that: without this var it logs "WebDAV client
|
||||
# unavailable" and silently writes no posters/backdrops, so every record then
|
||||
# fails meta-watch's poster gate. Drop only after re-tagging the plugin.
|
||||
WEBDAV_URL: "http://metacore-app:9000/webdav"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederprowlarr/tmdb-cache:/cache
|
||||
networks:
|
||||
- metafeederprowlarr
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config UI (set the Prowlarr key, TMDB token)
|
||||
# behind Yundera OIDC SSO. container_name == subdomain prefix. The gateway
|
||||
# still reaches the feeder directly on `pcs` (this gate is for the human only);
|
||||
# index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederprowlarr:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederprowlarr
|
||||
hostname: metafeederprowlarr
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederprowlarr-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederprowlarr-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederprowlarr-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederprowlarr-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederprowlarr-feeder
|
||||
networks:
|
||||
- metafeederprowlarr
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederprowlarr:
|
||||
name: metafeederprowlarr
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederprowlarr
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**A standalone torrent + usenet feeder for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Prowlarr bridges your **Prowlarr** indexers
|
||||
(Torznab/Newznab — torrent *and* usenet) into MetaMesh, so releases surface
|
||||
in **MetaWatch**. It's the torrent/usenet source tier, split out of
|
||||
MetaGateway so you can run only the feeders you want.
|
||||
|
||||
**Self-contained:** ships its own filename-parser + TMDB enrichment, so each
|
||||
release gets a clean title, poster and plot.
|
||||
|
||||
**Source:**
|
||||
- **Prowlarr** — Torznab/Newznab indexers, torrent or usenet (reached as
|
||||
`prowlarr` on the shared network).
|
||||
|
||||
**Want decentralized discovery too?** Install **MetaFeeder · Tribler** — the
|
||||
Tribler source now lives in its own app.
|
||||
|
||||
**Also requires:** **MetaGateway** — its meta-core stores the resolved
|
||||
records and announces them to the swarm. The gateway must list this feeder
|
||||
in its config.
|
||||
|
||||
fr_fr: |
|
||||
**Un feeder torrent + usenet autonome pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Prowlarr connecte vos indexeurs **Prowlarr**
|
||||
(Torznab/Newznab — torrent *et* usenet) à MetaMesh pour qu'ils apparaissent
|
||||
dans **MetaWatch**. C'est la couche de sources torrent/usenet, détachée de
|
||||
MetaGateway pour n'exécuter que les feeders souhaités.
|
||||
|
||||
**Autonome :** embarque ses propres plugins filename-parser + TMDB.
|
||||
|
||||
**Source :** l'app **Prowlarr** (indexeurs Torznab/Newznab, torrent ou
|
||||
usenet, atteinte comme `prowlarr`).
|
||||
|
||||
**Envie de découverte décentralisée ?** Installez **MetaFeeder · Tribler** —
|
||||
la source Tribler est désormais une app dédiée.
|
||||
|
||||
**Nécessite aussi :** l'app **MetaGateway** (son meta-core stocke et annonce
|
||||
les enregistrements).
|
||||
|
||||
tagline:
|
||||
en_us: Standalone Prowlarr/Torznab (torrent + usenet) feeder for MetaMesh
|
||||
fr_fr: Feeder Prowlarr/Torznab (torrent + usenet) autonome pour MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Prowlarr
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederProwlarr/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederProwlarr/thumbnail.png
|
||||
|
||||
store_app_id: metafeederprowlarr
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first, plus the **Prowlarr**
|
||||
store app (Torznab/Newznab).
|
||||
|
||||
**After installing this app:**
|
||||
1. In **Prowlarr**: complete its first-run wizard, add one or more
|
||||
indexers (torrent or usenet), then copy the **API Key**
|
||||
(Settings → General).
|
||||
2. Open this app's config UI (the dashboard link). Paste the Prowlarr key
|
||||
into the indexer's API-key field; optionally set a TMDB v4 token for
|
||||
posters. Save — the feeder reloads automatically.
|
||||
3. In **MetaGateway**, make sure this feeder is listed in its feeders
|
||||
config (`http://metafeederprowlarr-feeder:8080`) so it gets
|
||||
discovered.
|
||||
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederprowlarr/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeederprowlarr/tmdb-cache
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederprowlarr-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 136 KiB |
@@ -0,0 +1,47 @@
|
||||
# MetaFeeder · Prowlarr — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeederprowlarr-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
The feeder publishes its resolved records to the shared MetaCore over HTTP and writes its
|
||||
own service registration and plugin artwork through MetaCore's WebDAV endpoint. Those
|
||||
writes land in directories created and owned by MetaCore's root-running container; a
|
||||
uid-1000 process is refused. The feeder image also runs a small supervised process group
|
||||
(fetcher + enrichment plugin clients) that expects to be able to signal its children.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeederprowlarr/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
- The enrichment sidecars (`-filename-parser`, `-tmdb`) run **unprivileged** — no `user:`
|
||||
field, so Yundera starts them as `$PUID:$PGID`. `-filename-parser` is on the app-private
|
||||
`metafeederprowlarr` bridge only; `-tmdb` also joins `pcs` because MetaCore calls back into
|
||||
it, and it carries an app-prefixed `container_name` so it cannot collide with another
|
||||
app's service in the shared DNS namespace. Neither carries Caddy labels or publishes a
|
||||
host port.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Writes into MetaCore's root-owned `/meta-core` tree and through
|
||||
its WebDAV endpoint fail, and the feeder starts but silently publishes nothing.
|
||||
- **Have MetaCore relax the ownership of its shared directory.** That would loosen
|
||||
permissions for every MetaMesh app at once, which is strictly worse than one root
|
||||
container with no user-data mounts.
|
||||
- **Drop privileges after startup inside the image.** Upstream change; tracked, not
|
||||
available in the shipped image.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state — configuration, cursor position, and
|
||||
(where applicable) the TMDB cache — lives under `/DATA/AppData/metafeederprowlarr/`, which is the unit
|
||||
Maison archives on uninstall and restores from backup, so an uninstall/reinstall with "keep
|
||||
user data" resumes where it left off rather than re-indexing from scratch.
|
||||
|
After Width: | Height: | Size: 136 KiB |
@@ -0,0 +1,166 @@
|
||||
name: metafeederscihub
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Sci-Hub — papers resolved through a Sci-Hub mirror.
|
||||
#
|
||||
# Opt-in: soft-skips (healthy, answers nothing) until SCIHUB_MIRRORS names at
|
||||
# least one mirror. Split out of the former MetaFeeder · Papers app.
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeederscihub-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeederscihub-*` prefix + the per-stack `metafeederscihub` network +
|
||||
# /DATA/AppData/metafeederscihub/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederscihub-feeder — the scihub feeder. Needs SCIHUB_MIRRORS to do anything.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederscihub-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-scihub:1.0.1
|
||||
cpu_shares: 30
|
||||
container_name: metafeederscihub-feeder
|
||||
hostname: metafeederscihub-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
# Opt-in. Uncomment + set mirrors to enable; blank → the feeder soft-skips.
|
||||
# SCIHUB_MIRRORS: "https://sci-hub.se,https://sci-hub.st"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederscihub/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeederscihub
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederscihub:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederscihub
|
||||
hostname: metafeederscihub
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederscihub-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederscihub-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederscihub-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederscihub-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederscihub-feeder
|
||||
networks:
|
||||
- metafeederscihub
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederscihub:
|
||||
name: metafeederscihub
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederscihub
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Sci-Hub for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Sci-Hub resolves papers through one or more **Sci-Hub** mirrors you
|
||||
configure. Without a mirror it stays healthy and simply finds nothing.
|
||||
|
||||
Sci-Hub distributes papers without publisher permission; whether using it is
|
||||
lawful where you run this app is your responsibility.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**Sci-Hub pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Sci-Hub résout des articles via un ou plusieurs miroirs **Sci-Hub**
|
||||
que vous configurez. Sans miroir, il reste sain et ne trouve rien.
|
||||
|
||||
Sci-Hub diffuse des articles sans l'autorisation des éditeurs ; la légalité de
|
||||
son usage là où vous hébergez cette app relève de votre responsabilité.
|
||||
**Nécessite** l'app **MetaGateway**.
|
||||
tagline:
|
||||
en_us: Papers via Sci-Hub mirrors you configure, for MetaMesh
|
||||
fr_fr: Articles via les miroirs Sci-Hub que vous configurez, pour MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Sci-Hub
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederSciHub/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederSciHub/thumbnail.png
|
||||
|
||||
store_app_id: metafeederscihub
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores and
|
||||
announces what this feeder finds).
|
||||
|
||||
**After installing this app:** in **MetaGateway**, add this feeder from the
|
||||
dashboard's *Add a feeder* box — name `scihub-feeder`, URL `http://metafeederscihub-feeder:8080`.
|
||||
The gateway restarts itself: feeder discovery is startup-only.
|
||||
|
||||
**Mirrors:** set `SCIHUB_MIRRORS` (comma-separated URLs) in this app's
|
||||
environment, then restart the app. Without it the feeder soft-skips.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederscihub/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederscihub-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 132 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · Sci-Hub — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeederscihub-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Papers), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeederscihub/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeederscihub/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 132 KiB |
@@ -0,0 +1,238 @@
|
||||
name: metafeedersuwayomi
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Suwayomi — the comics/manga CHAPTER tier for MetaMesh.
|
||||
#
|
||||
# Turns a work into actual chapters by searching the operator's Suwayomi, and
|
||||
# (since 1.0.3) mints its own work cards for free-text search, so it works on its
|
||||
# own. Split out of the former MetaFeeder · Comic app; **MetaFeeder · AniList**
|
||||
# is the optional identity tier beside it.
|
||||
#
|
||||
# ⚠ **Needs meta-gateway ≥ 1.0.28** — chapters arrive unresolved and their page
|
||||
# list follows as an EnrichPatch, which an older gateway drops.
|
||||
#
|
||||
# ⚠ **Suwayomi itself is a SEPARATE app — install it first.** This feeder ships
|
||||
# pointed at `suwayomi:4567`, the container the **Suwayomi** store app publishes
|
||||
# on the shared `pcs` network.
|
||||
#
|
||||
# ⚠ **Querying Suwayomi WRITES.** Its search and chapter calls are GraphQL
|
||||
# mutations that cache source results into its library database.
|
||||
#
|
||||
# ⚠ **Suwayomi must be reachable UNAUTHENTICATED.** An undownloaded chapter is a
|
||||
# list of per-page URLs pointing at the Suwayomi container, redeemed by the
|
||||
# READER's meta-share peer with no credentials. Suwayomi's `simple_login` 401s
|
||||
# those routes — a library that lists every chapter and opens none. The Suwayomi
|
||||
# store app therefore runs `AUTH_MODE: none` behind an AppShield/OIDC gate.
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeedersuwayomi-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeedersuwayomi-*` prefix + the per-stack `metafeedersuwayomi` network +
|
||||
# /DATA/AppData/metafeedersuwayomi/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeedersuwayomi-feeder — the chapter tier. Resolves a work to titles,
|
||||
# searches every installed source, returns the chapters. Chapters are listed
|
||||
# lazily: each chapter's page list is fetched only when a reader opens it.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedersuwayomi-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-suwayomi:1.0.4
|
||||
cpu_shares: 30
|
||||
container_name: metafeedersuwayomi-feeder
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeedersuwayomi-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info,suwayomi_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
# INFRA-ONLY env, and only a FIRST-BOOT SEED: `base_url` is a field on
|
||||
# this feeder's own config page (the dashboard link), and a value saved
|
||||
# there wins. `suwayomi` is the container the Suwayomi store app publishes
|
||||
# on `pcs` — change it only to point at a Suwayomi somewhere else, and see
|
||||
# the reachability warning in the header before you do.
|
||||
SUWAYOMI_URL: "http://suwayomi:4567"
|
||||
# Listing chapters is cheap (one call returns them all); RESOLVING a
|
||||
# chapter's page list is what costs. 0 = fetch nothing until a reader
|
||||
# clicks. Raise it to pre-warm the first N ordinals per work.
|
||||
SUWAYOMI_WARM_CHAPTERS: "0"
|
||||
# ⚠ SUWAYOMI_MAX_CHAPTERS is deliberately NOT set: it bounds how many
|
||||
# chapters are LISTED, and any value can only truncate a long series into
|
||||
# looking short.
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedersuwayomi/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeedersuwayomi
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedersuwayomi:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeedersuwayomi
|
||||
hostname: metafeedersuwayomi
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeedersuwayomi-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeedersuwayomi-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeedersuwayomi-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeedersuwayomi-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeedersuwayomi-feeder
|
||||
networks:
|
||||
- metafeedersuwayomi
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeedersuwayomi:
|
||||
name: metafeedersuwayomi
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeedersuwayomi
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**The comics and manga chapter tier for MetaMesh — what gives MetaRead
|
||||
something to read.**
|
||||
|
||||
MetaFeeder · Suwayomi supplies the *chapters*, from whichever source extensions
|
||||
you installed in the **Suwayomi** app, and mints its own work cards so search
|
||||
works on its own. Add **MetaFeeder · AniList** to name works across sources.
|
||||
|
||||
**Install Suwayomi first.** It is a separate app in this store (with a
|
||||
FlareSolverr sidecar, so Cloudflare-protected sources work). This feeder arrives
|
||||
pointed at it and needs no wiring.
|
||||
|
||||
**You own the sources.** Suwayomi ships exactly one built-in source — *Local
|
||||
source*, which reads CBZ/CBR from its own folder. Everything else is an extension
|
||||
**you** install through the Suwayomi app's own UI. That keeps the scraping and the
|
||||
legal surface outside MetaMesh, exactly as **MetaFeeder · Prowlarr** keeps
|
||||
indexers outside it.
|
||||
|
||||
**Multi-source by design.** Every installed source that carries a work
|
||||
contributes, and each chapter row offers a chooser across them.
|
||||
|
||||
**Lazy by default.** Listing a series' chapters costs one upstream call however
|
||||
long it is; a chapter's page list is fetched only when someone opens it.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
**Pairs with: MetaRead**, the reading client.
|
||||
fr_fr: |
|
||||
**La couche chapitres BD et manga de MetaMesh — ce qui donne à MetaRead
|
||||
quelque chose à lire.**
|
||||
|
||||
MetaFeeder · Suwayomi fournit les *chapitres*, depuis les extensions de sources
|
||||
installées dans l'app **Suwayomi**, et crée ses propres fiches d'œuvres pour que
|
||||
la recherche fonctionne seule. Ajoutez **MetaFeeder · AniList** pour nommer les
|
||||
œuvres entre sources.
|
||||
|
||||
**Vous possédez les sources.** Suwayomi n'embarque que *Local source* ; tout le
|
||||
reste est une extension que **vous** installez via l'interface Suwayomi.
|
||||
|
||||
**Paresseux par défaut.** Lister les chapitres coûte un appel ; les pages d'un
|
||||
chapitre ne sont récupérées qu'à l'ouverture.
|
||||
|
||||
**Nécessite :** les apps **MetaGateway** et **Suwayomi**. **Va avec : MetaRead.**
|
||||
tagline:
|
||||
en_us: Suwayomi — the comics and manga chapter tier for MetaMesh
|
||||
fr_fr: Suwayomi — la couche chapitres BD et manga de MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Suwayomi
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederSuwayomi/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederSuwayomi/thumbnail.png
|
||||
|
||||
store_app_id: metafeedersuwayomi
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** **MetaGateway** first, then **Suwayomi** — the separate
|
||||
store app this feeder reads. This app arrives pointed at `suwayomi:4567`; without
|
||||
that app installed it stays healthy and simply finds no chapters.
|
||||
|
||||
**After installing this app:**
|
||||
1. In **MetaGateway**, add this feeder from the dashboard's *Add a feeder* box —
|
||||
name `suwayomi-feeder`, URL `http://metafeedersuwayomi-feeder:8080`. The
|
||||
gateway restarts itself: feeder discovery is **startup-only**.
|
||||
2. Open the **Suwayomi** app and install the source extensions you want, under
|
||||
*Extensions*. Nothing is installed by default.
|
||||
3. Optionally drop `.cbz` / `.cbr` files into
|
||||
`/DATA/AppData/suwayomi/data/local/<Series Name>/` — the built-in *Local
|
||||
source* picks them up with no extension at all.
|
||||
4. Optionally install **MetaFeeder · AniList** so chapters from several sources
|
||||
converge on one work page.
|
||||
|
||||
This app's own dashboard link opens the **feeder's config page** (Suwayomi URL,
|
||||
chapters per work, chapters pre-loaded) — not Suwayomi.
|
||||
|
||||
**⚠ Querying Suwayomi writes to it.** Rows will appear in its library that you
|
||||
did not add by hand. That is inherent to how the aggregator works.
|
||||
|
||||
**Suwayomi's own login must stay off** — the reader fetches chapter pages from it
|
||||
unauthenticated, so enabling it gives a library that lists every chapter and can
|
||||
open none.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeedersuwayomi/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeedersuwayomi-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 133 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · Suwayomi — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeedersuwayomi-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Comic), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeedersuwayomi/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeedersuwayomi/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 133 KiB |
@@ -0,0 +1,263 @@
|
||||
name: metafeedertmdb
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · TMDB — the CARD TIER feeder for MetaMesh.
|
||||
#
|
||||
# A "card" is a metadata-only record identifying a *work* (a series, a film) —
|
||||
# title, poster, description and an id bag (tmdb/imdb/…). It has no bytes and by
|
||||
# design never will. This feeder resolves cards from **TMDB** so a free-text
|
||||
# search in **MetaWatch** returns a clean poster grid of *works* instead of a
|
||||
# wall of raw release names.
|
||||
#
|
||||
# Why it is its own tier: search is split in two halves.
|
||||
# Phase 1 — CARD (this app): <free text> → one cached TMDB search
|
||||
# → a poster grid of works
|
||||
# Phase 2 — RETRIEVAL (indexer / usenet / tribler feeders): the user clicks a
|
||||
# card → the full indexer budget is spent on
|
||||
# that ONE work → the episode / version list.
|
||||
# Previously a single query resolved ten speculative candidate works *and*
|
||||
# searched every indexer for all of them against one shared rate bucket, so the
|
||||
# work the user actually wanted got a tenth of the budget. Installing this app
|
||||
# is what makes the first half cheap.
|
||||
#
|
||||
# Cheap and self-contained: it calls TMDB directly, holds no bytes, runs no
|
||||
# enrichment sidecars, and never touches an indexer. It also does NOT talk to
|
||||
# meta-core itself — the MetaGateway core stores and announces the cards it
|
||||
# returns.
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeedertmdb-feeder:8080`) in its feeders config so it gets
|
||||
# discovered. Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeedertmdb-*` prefix + the per-stack
|
||||
# `metafeedertmdb` network + /DATA/AppData/metafeedertmdb/ data root, so it
|
||||
# installs alongside the other Meta* apps with zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeedertmdb-feeder — the feeder (meta-feeder-tmdb) hosting the `tmdb`
|
||||
# upstream. The gateway core reaches it directly on `pcs` for
|
||||
# discovery/query/compute; the human configures it through the AppShield gate
|
||||
# below.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedertmdb-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-tmdb:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeedertmdb-feeder
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM instead of
|
||||
# swap. This feeder holds no bytes and does far less than the indexer feeder
|
||||
# (one TMDB call + a redb cache), so this is generous, not tight.
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeedertmdb-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# INFRA-ONLY env (the "config: no env if in webui" rule). Operator config —
|
||||
# TMDB token, rate/burst, cards-per-search — is read from the persisted
|
||||
# config.json written by this feeder's config page (behind the AppShield
|
||||
# gate below). TMDB_TOKEN / TMDB_RATE_PER_SEC / TMDB_BURST / CARD_TOP_N
|
||||
# exist only as deprecated first-boot seeds; don't set them here.
|
||||
# META_FEEDER_STATE_DIR is left at its default (/data/meta-feeder), which
|
||||
# is what the volume below mounts.
|
||||
RUST_LOG: "info,meta_feeder=info,card_feeder=info"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config (config.json saved via the config UI) + the redb
|
||||
# TMDB response cache. Wiping the cache is safe but re-burns TMDB budget
|
||||
# on every card the swarm re-discovers.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedertmdb/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeedertmdb
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config UI (set the TMDB token, tune the rate
|
||||
# and cards-per-search) behind Yundera OIDC SSO. container_name == subdomain
|
||||
# prefix. The gateway still reaches the feeder directly on `pcs` (this gate is
|
||||
# for the human only); index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedertmdb:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeedertmdb
|
||||
hostname: metafeedertmdb
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeedertmdb-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeedertmdb-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeedertmdb-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeedertmdb-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeedertmdb-feeder
|
||||
networks:
|
||||
- metafeedertmdb
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeedertmdb:
|
||||
name: metafeedertmdb
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeedertmdb
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**The card tier — the half of MetaMesh search that makes it feel like a
|
||||
streaming app.**
|
||||
|
||||
MetaFeeder · TMDB resolves **cards** from **TMDB**: metadata-only records
|
||||
identifying a *work* — a series or a film — with title, poster, plot and a
|
||||
bag of cross-source ids. A card carries no bytes. Its job is to turn a
|
||||
free-text search in **MetaWatch** into a clean **poster grid of works**
|
||||
instead of a wall of raw release names.
|
||||
|
||||
**Why it exists.** Search is split in two:
|
||||
|
||||
1. **Card** (this app) — one cached TMDB lookup → a grid of works. Costs
|
||||
no indexer requests at all.
|
||||
2. **Retrieval** (**MetaFeeder · Prowlarr**, **· Usenet**, **· Tribler**) —
|
||||
you click a card, and the *entire* indexer budget is spent on that one
|
||||
work → its episode and version list.
|
||||
|
||||
Before the split, a single query speculatively resolved ten candidate works
|
||||
*and* searched every indexer for all of them against one shared rate
|
||||
bucket — so the work you actually wanted got a tenth of the budget.
|
||||
|
||||
**Cheap and self-contained:** it calls TMDB directly, holds no bytes, runs
|
||||
no enrichment sidecars, and never touches an indexer. ~256 MB RAM.
|
||||
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved cards and
|
||||
announces them to the swarm. The gateway must list this feeder in its
|
||||
config. You also need a free **TMDB** read token.
|
||||
|
||||
**Pairs with:** any retrieval feeder (**Indexer**, **Usenet**, **Tribler**)
|
||||
— cards are what you click *before* those do their work.
|
||||
|
||||
fr_fr: |
|
||||
**La couche « card » — la moitié de la recherche MetaMesh qui lui donne des
|
||||
allures d'app de streaming.**
|
||||
|
||||
MetaFeeder · TMDB résout des **cards** depuis **TMDB** : des enregistrements
|
||||
purement métadonnées identifiant une *œuvre* — une série, un film — avec
|
||||
titre, affiche, synopsis et un jeu d'identifiants inter-sources. Une card ne
|
||||
contient aucun octet. Son rôle : transformer une recherche libre dans
|
||||
**MetaWatch** en une **grille d'affiches d'œuvres** plutôt qu'en un mur de
|
||||
noms de releases bruts.
|
||||
|
||||
**Pourquoi.** La recherche est scindée en deux :
|
||||
|
||||
1. **Card** (cette app) — une requête TMDB mise en cache → une grille
|
||||
d'œuvres. Ne consomme aucune requête d'indexeur.
|
||||
2. **Récupération** (**MetaFeeder · Prowlarr**, **· Usenet**, **· Tribler**)
|
||||
— vous cliquez une card, et *tout* le budget d'indexation est dépensé sur
|
||||
cette seule œuvre → la liste de ses épisodes et versions.
|
||||
|
||||
Auparavant, une seule requête résolvait spéculativement dix œuvres
|
||||
candidates *et* interrogeait tous les indexeurs pour chacune, sur un même
|
||||
seau de débit — l'œuvre réellement voulue n'obtenait qu'un dixième du
|
||||
budget.
|
||||
|
||||
**Léger et autonome :** appelle TMDB directement, ne stocke aucun octet,
|
||||
aucun plugin d'enrichissement, ne touche jamais un indexeur. ~256 Mo de RAM.
|
||||
|
||||
**Nécessite :** l'app **MetaGateway** (son meta-core stocke et annonce les
|
||||
cards) et un jeton de lecture **TMDB** gratuit.
|
||||
|
||||
tagline:
|
||||
en_us: TMDB card tier — turns MetaMesh search into a poster grid of works
|
||||
fr_fr: Couche « card » TMDB — transforme la recherche MetaMesh en grille d'affiches
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · TMDB
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTMDB/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTMDB/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTMDB/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTMDB/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTMDB/thumbnail.png
|
||||
|
||||
store_app_id: metafeedertmdb
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores
|
||||
and announces the cards this feeder resolves).
|
||||
|
||||
**After installing this app:**
|
||||
1. Get a free **TMDB v4 read access token** — themoviedb.org →
|
||||
Settings → API. (The v4 "Read Access Token", not the v3 API key.)
|
||||
2. Open this app's config UI (the dashboard link) and paste it into
|
||||
**TMDB v4 token**. Leave the rate, burst and cards-per-search fields
|
||||
blank unless you have a reason — the defaults (20 req/s, burst 20,
|
||||
10 cards per search) are sane. Save, then restart this app: the
|
||||
feeder reads its config at startup, there is no hot reload.
|
||||
3. In **MetaGateway**, add this feeder to its feeders config —
|
||||
name `tmdb-feeder`, URL `http://metafeedertmdb-feeder:8080` — from
|
||||
the gateway dashboard's *Add a feeder* box. The gateway restarts
|
||||
itself and picks up the `tmdb` upstream (it will show **LIVE**, with
|
||||
the tags CARD / MOVIE / SERIES).
|
||||
|
||||
**Without a TMDB token** the feeder stays healthy but soft-skips: card
|
||||
searches simply return nothing. That is by design, not a failure.
|
||||
|
||||
**Cards vs. results.** This app only produces the poster grid. To
|
||||
actually get episodes and streams behind a card you also need a
|
||||
retrieval feeder — **MetaFeeder · Prowlarr**, **· Usenet** or
|
||||
**· Tribler**.
|
||||
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeedertmdb/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeedertmdb-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,41 @@
|
||||
# MetaFeeder · TMDBs — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeedertmdb-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
The feeder publishes its resolved records to the shared MetaCore over HTTP and writes its
|
||||
own service registration and plugin artwork through MetaCore's WebDAV endpoint. Those
|
||||
writes land in directories created and owned by MetaCore's root-running container; a
|
||||
uid-1000 process is refused. The feeder image also runs a small supervised process group
|
||||
(fetcher + enrichment plugin clients) that expects to be able to signal its children.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeedertmdb/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Writes into MetaCore's root-owned `/meta-core` tree and through
|
||||
its WebDAV endpoint fail, and the feeder starts but silently publishes nothing.
|
||||
- **Have MetaCore relax the ownership of its shared directory.** That would loosen
|
||||
permissions for every MetaMesh app at once, which is strictly worse than one root
|
||||
container with no user-data mounts.
|
||||
- **Drop privileges after startup inside the image.** Upstream change; tracked, not
|
||||
available in the shipped image.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state — configuration, cursor position, and
|
||||
(where applicable) the TMDB cache — lives under `/DATA/AppData/metafeedertmdb/`, which is the unit
|
||||
Maison archives on uninstall and restores from backup, so an uninstall/reinstall with "keep
|
||||
user data" resumes where it left off rather than re-indexing from scratch.
|
||||
|
After Width: | Height: | Size: 1.0 MiB |
|
After Width: | Height: | Size: 90 KiB |
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,314 @@
|
||||
name: metafeedertribler
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Tribler — a STANDALONE, self-contained Tribler feeder for MetaMesh.
|
||||
#
|
||||
# The tribler source was extracted from the torrent feeder into its own image
|
||||
# (`meta-feeder-tribler`), so this is a single-plugin feeder. It bundles its OWN
|
||||
# headless Tribler core (no separate Tribler app needed) and bridges the
|
||||
# decentralized IPv8 Tribler network into MetaMesh, so releases surface in
|
||||
# **MetaWatch**. Mirrors the meta-gateway dev stack
|
||||
# (`dev/docker-compose.feeder-tribler.yml`).
|
||||
#
|
||||
# Self-contained for enrichment: ships its own filename-parser + tmdb plugins,
|
||||
# so each discovered release gets a clean title, poster and plot.
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the shared **MetaCore** app (`metacore-app:9000`) — where resolved records
|
||||
# + TMDB posters are stored; the shared **MetaShare** app then announces
|
||||
# them to the swarm.
|
||||
# The **MetaGateway** app must list this feeder's URL
|
||||
# (`http://metafeedertribler-feeder:8080`) in its feeders config so it
|
||||
# discovers + queries it (MetaGateway, MetaCore and this feeder share `pcs`).
|
||||
#
|
||||
# Everything is scoped under the `metafeedertribler-*` prefix + the per-stack
|
||||
# `metafeedertribler` network + /DATA/AppData/metafeedertribler/ data root, so it
|
||||
# installs alongside the other Meta* apps with zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# tribler-instance — the bundled headless Tribler core (IPv8 overlay). Private
|
||||
# to this stack (metafeedertribler network only), reached by the feeder at its
|
||||
# default sidecar URL `http://tribler-instance:8085`. A complete baked config
|
||||
# (api on 0.0.0.0:8085, key `changeme`) is seeded on first boot from the
|
||||
# pre-install-cmd-written seed file. IPv8 overlay on UDP 8090.
|
||||
# ---------------------------------------------------------------------------
|
||||
tribler-instance:
|
||||
# Pinned: v8.4.2 is the digest `:latest` resolved to when this was pinned
|
||||
# (sha256:e5f0ff1f…), so this is a tag change only, not a version bump.
|
||||
image: ghcr.io/tribler/tribler:v8.4.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeedertribler-instance
|
||||
hostname: tribler-instance
|
||||
restart: unless-stopped
|
||||
entrypoint: ["/bin/sh", "-c"]
|
||||
command:
|
||||
- |
|
||||
set -e
|
||||
CFG=/root/.Tribler/8.0/configuration.json
|
||||
if [ ! -f "$$CFG" ]; then
|
||||
mkdir -p "$$(dirname "$$CFG")"
|
||||
cp /seed/configuration.json "$$CFG"
|
||||
echo "tribler: installed baked config at $$CFG"
|
||||
fi
|
||||
exec /usr/share/tribler/tribler -s
|
||||
environment:
|
||||
LANG: "C.UTF-8"
|
||||
expose:
|
||||
- 8085
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedertribler/tribler-state:/root/.Tribler
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedertribler/tribler-seed/configuration.json:/seed/configuration.json:ro
|
||||
networks:
|
||||
- metafeedertribler
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeedertribler-feeder — the tribler feeder (dedicated meta-feeder-tribler
|
||||
# image). A thin client of tribler-instance; self-publishes resolved records to
|
||||
# the shared meta-core + drives the bundled enrichment plugins. The gateway
|
||||
# reaches it directly on `pcs`; the human configures it through the AppShield
|
||||
# gate below.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedertribler-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-tribler:1.0.4
|
||||
cpu_shares: 30
|
||||
container_name: metafeedertribler-feeder
|
||||
hostname: metafeedertribler-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# INFRA-ONLY env (the "config: no env if in webui" rule). Operator config —
|
||||
# Tribler sidecar URL, api key, TMDB token / language, meta-core URL — is
|
||||
# read from the persisted config.json (pre-seeded by pre-install-cmd, then
|
||||
# editable in the config UI behind the AppShield gate). These env vars are
|
||||
# wiring only: enrichment sidecars, the enrich callback, the peer id
|
||||
# (analytics group by it), and the state dir.
|
||||
RUST_LOG: "info,meta_feeder=info,tribler_feeder=info"
|
||||
META_GATEWAY_PEER_ID: "metafeedertribler"
|
||||
FILENAME_PARSER_URL: "http://metafeedertribler-filename-parser:8080"
|
||||
TMDB_PLUGIN_URL: "http://metafeedertribler-tmdb:8080"
|
||||
META_FEEDER_CALLBACK_URL: "http://metafeedertribler-feeder:8080/enrich/callback"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config (config.json pre-seeded + editable via UI) + redb
|
||||
# cache. config.json lives at gateway/tribler/config.json under this dir.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedertribler/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
depends_on:
|
||||
# service_started (not _healthy): tribler core + the plugin images ship no
|
||||
# healthcheck (curl absent in the plugins; tribler boots slowly).
|
||||
tribler-instance:
|
||||
condition: service_started
|
||||
metafeedertribler-filename-parser:
|
||||
condition: service_started
|
||||
metafeedertribler-tmdb:
|
||||
condition: service_started
|
||||
networks:
|
||||
- metafeedertribler
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bundled enrichment plugins (self-contained). filename-parser per file; tmdb
|
||||
# once per release — tmdb writes posters/backdrops to the shared meta-core's
|
||||
# WebDAV (over pcs) so the ingester seeds them. No healthcheck: these plugin
|
||||
# images don't ship one (curl absent).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedertribler-filename-parser:
|
||||
image: ghcr.io/worph/metamesh-plugin-filename-parser:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeedertribler-filename-parser
|
||||
hostname: metafeedertribler-filename-parser
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
PORT: "8080"
|
||||
EMIT_TITLE_TAGS: "true"
|
||||
networks:
|
||||
- metafeedertribler
|
||||
|
||||
metafeedertribler-tmdb:
|
||||
image: ghcr.io/worph/metamesh-plugin-tmdb:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeedertribler-tmdb
|
||||
hostname: metafeedertribler-tmdb
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
PORT: "8080"
|
||||
CACHE_DIR: "/cache"
|
||||
# Posters land in the shared meta-core (over pcs) so the ingester seeds them.
|
||||
WEBDAV_URL: "http://metacore-app:9000/webdav"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeedertribler/tmdb-cache:/cache
|
||||
networks:
|
||||
- metafeedertribler
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config UI (Tribler sidecar URL / api key, TMDB
|
||||
# token) behind Yundera OIDC SSO. container_name == subdomain prefix. The
|
||||
# gateway still reaches the feeder directly on `pcs` (this gate is for the
|
||||
# human only); index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeedertribler:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeedertribler
|
||||
hostname: metafeedertribler
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeedertribler-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeedertribler-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeedertribler-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeedertribler-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeedertribler-feeder
|
||||
networks:
|
||||
- metafeedertribler
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeedertribler:
|
||||
name: metafeedertribler
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeedertribler
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**A standalone, self-contained Tribler feeder for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Tribler bridges the decentralized **Tribler** network into
|
||||
MetaMesh, so releases surface in **MetaWatch**. It bundles its own headless
|
||||
Tribler core — no separate Tribler app to install — for anonymous IPv8
|
||||
torrent discovery, with no indexer keys required. Split out of MetaGateway
|
||||
so you can run only the feeders you want.
|
||||
|
||||
**Self-contained:** ships its own Tribler core + filename-parser + TMDB
|
||||
enrichment, so each release gets a clean title, poster and plot.
|
||||
|
||||
**Also requires:** **MetaGateway** — its meta-core stores the resolved
|
||||
records and announces them to the swarm. The gateway must list this feeder
|
||||
in its config.
|
||||
|
||||
fr_fr: |
|
||||
**Un feeder Tribler autonome et complet pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Tribler connecte le réseau décentralisé **Tribler** à MetaMesh
|
||||
pour que les releases apparaissent dans **MetaWatch**. Il embarque son
|
||||
propre cœur Tribler headless — aucune app Tribler séparée à installer — pour
|
||||
une découverte de torrents anonyme via IPv8, sans clé d'indexeur. Détaché de
|
||||
MetaGateway pour n'exécuter que les feeders souhaités.
|
||||
|
||||
**Autonome :** embarque son cœur Tribler + ses plugins filename-parser + TMDB.
|
||||
|
||||
**Nécessite aussi :** l'app **MetaGateway** (son meta-core stocke et annonce
|
||||
les enregistrements).
|
||||
|
||||
tagline:
|
||||
en_us: Self-contained Tribler feeder (bundled core) for the MetaMesh network
|
||||
fr_fr: Feeder Tribler autonome (cœur inclus) pour le réseau MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Tribler
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTribler/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederTribler/thumbnail.png
|
||||
|
||||
store_app_id: metafeedertribler
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (this feeder publishes to
|
||||
its shared meta-core). The Tribler core is bundled — nothing else to add.
|
||||
|
||||
**After installing this app:**
|
||||
1. It works out of the box: the feeder is pre-wired to its bundled Tribler
|
||||
core. Optionally open the config UI (the dashboard link) to set a TMDB
|
||||
v4 token for posters, then Save — the feeder reloads automatically.
|
||||
2. In **MetaGateway**, make sure this feeder is listed in its feeders
|
||||
config (`http://metafeedertribler-feeder:8080`) so it gets discovered
|
||||
(or add it from the gateway dashboard).
|
||||
|
||||
**Note:** the bundled Tribler core needs a few minutes to bootstrap onto
|
||||
the IPv8 network before search results appear.
|
||||
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
pre-install-cmd: |
|
||||
# Seed the bundled Tribler core config (api on 0.0.0.0:8085, key `changeme`).
|
||||
if [ ! -f /DATA/AppData/metafeedertribler/tribler-seed/configuration.json ]; then
|
||||
cat > /DATA/AppData/metafeedertribler/tribler-seed/configuration.json <<'TRIBLER_CFG'
|
||||
{"api":{"http_enabled":true,"http_port":8085,"http_host":"0.0.0.0","https_enabled":false,"https_host":"127.0.0.1","https_port":0,"https_certfile":"https_certfile","http_port_running":0,"https_port_running":0,"key":"changeme"},"headless":false,"start_minimized":false,"ipv8":{"interfaces":[{"interface":"UDPIPv4","ip":"0.0.0.0","port":8090},{"interface":"UDPIPv6","ip":"::","port":8091}],"keys":[{"alias":"anonymous id","generation":"curve25519","file":"/root/.Tribler/ec_multichain.pem"},{"alias":"secondary","generation":"curve25519","file":"/root/.Tribler/secondary_key.pem"}],"logger":{"level":"INFO"},"working_directory":"/root/.Tribler","walker_interval":0.5,"overlays":[{"class":"DiscoveryCommunity","key":"anonymous id","walkers":[{"strategy":"RandomWalk","peers":20,"init":{"timeout":3}},{"strategy":"RandomChurn","peers":-1,"init":{"sample_size":8,"ping_interval":10,"inactive_time":27.5,"drop_time":57.5}},{"strategy":"PeriodicSimilarity","peers":-1,"init":{}}],"bootstrappers":[{"class":"DispersyBootstrapper","init":{"ip_addresses":[["130.161.119.206",6421],["130.161.119.206",6422],["131.180.27.155",6423],["131.180.27.156",6424],["131.180.27.161",6427],["131.180.27.161",6521],["131.180.27.161",6522],["131.180.27.162",6523],["131.180.27.162",6524],["130.161.119.215",6525],["130.161.119.215",6526],["130.161.119.201",6527],["130.161.119.201",6528]],"dns_addresses":[["dispersy1.tribler.org",6421],["dispersy1.st.tudelft.nl",6421],["dispersy2.tribler.org",6422],["dispersy2.st.tudelft.nl",6422],["dispersy3.tribler.org",6423],["dispersy3.st.tudelft.nl",6423],["dispersy4.tribler.org",6424]],"bootstrap_timeout":30}}],"initialize":{},"on_start":[]}]},"statistics":false,"content_discovery_community":{"enabled":true},"database":{"enabled":true},"dht_discovery":{"enabled":true},"libtorrent":{"socks_listen_ports":[0,0,0,0,0],"listen_interface":"0.0.0.0","port":0,"proxy_type":0,"proxy_server":"","proxy_auth":"","max_connections_download":-1,"max_download_rate":0,"max_upload_rate":0,"use_advanced_rate_limits":false,"utp":true,"dht":true,"dht_readiness_timeout":30,"upnp":true,"natpmp":true,"lsd":true,"announce_to_all_tiers":false,"announce_to_all_trackers":false,"max_concurrent_http_announces":50,"check_after_complete":false,"download_defaults":{"anonymity_enabled":true,"number_hops":1,"safeseeding_enabled":true,"saveas":"/root/Downloads","seeding_mode":"forever","seeding_ratio":2,"seeding_time":60,"channel_download":false,"add_download_to_channel":false,"trackers_file":"","torrent_folder":"","auto_managed":false,"completed_dir":""},"active_downloads":3,"active_seeds":5,"active_checking":1,"active_dht_limit":88,"active_tracker_limit":1600,"active_lsd_limit":60,"active_limit":500,"ask_download_settings":false,"clear_orphaned_parts":false},"recommender":{"enabled":true},"rendezvous":{"enabled":true},"rss":{"enabled":true,"urls":[]},"torrent_checker":{"enabled":true},"tunnel_community":{"enabled":true,"min_circuits":3,"max_circuits":8},"versioning":{"enabled":true,"allow_pre":false},"watch_folder":{"enabled":false,"directory":"","check_interval":10},"state_dir":"/root/.Tribler","memory_db":false,"tray_icon_color":"","ui":{}}
|
||||
TRIBLER_CFG
|
||||
fi
|
||||
# Seed the feeder config so it points at the bundled Tribler core out of the
|
||||
# box (operator can override + add a TMDB token in the config UI).
|
||||
if [ ! -f /DATA/AppData/metafeedertribler/state/gateway/tribler/config.json ]; then
|
||||
cat > /DATA/AppData/metafeedertribler/state/gateway/tribler/config.json <<'FEEDER_CFG'
|
||||
{"meta_core_url":"http://metacore-app:9000","sidecar_url":"http://tribler-instance:8085","api_key":"changeme"}
|
||||
FEEDER_CFG
|
||||
fi
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeedertribler/state/gateway/tribler
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeedertribler/tmdb-cache
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeedertribler/tribler-state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeedertribler/tribler-seed
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeedertribler-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,55 @@
|
||||
# MetaFeeder · Tribler — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metafeedertribler-feeder` runs `user: 0:0`.
|
||||
2. `tribler-instance` is seeded with a Tribler core API key of `changeme`.
|
||||
3. `tribler-instance` runs the upstream Tribler image, which writes its state as root
|
||||
inside `/root/.Tribler`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. The feeder publishes resolved records to the shared MetaCore and writes plugin artwork
|
||||
through its WebDAV endpoint; those targets are created and owned by MetaCore's
|
||||
root-running container.
|
||||
2. Tribler's core exposes a localhost-style REST API and requires *some* key. The seeded
|
||||
value is a placeholder that matches the one baked into the feeder's own config, so the
|
||||
two agree out of the box and the app works immediately after installation — the
|
||||
functionality requirement in the guidelines. It is not a user credential and grants no
|
||||
access to anything beyond this Tribler core.
|
||||
3. Upstream image behaviour; the state directory is bind-mounted so it survives container
|
||||
recreation.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- **`tribler-instance` is not on the shared `pcs` network.** It is attached to the
|
||||
app-private `metafeedertribler` bridge only, so the API on `:8085` is reachable by this
|
||||
app's own feeder and by nothing else on the box — not by another store app, and not from
|
||||
the internet. The `changeme` key never crosses a network boundary a stranger can reach.
|
||||
- The feeder's configuration UI is behind AppShield + Authelia SSO
|
||||
(`ghcr.io/yundera/appshield:2.0.9`); the feeder itself carries no Caddy labels and
|
||||
publishes no host port.
|
||||
- No user directories are mounted — no `/DATA/Documents`, `/DATA/Downloads`, `/DATA/Media`
|
||||
or `/DATA/Gallery`. All binds are under `/DATA/AppData/metafeedertribler/`, declared in
|
||||
`x-compose-app.folders`.
|
||||
- Both config seeds in `pre-install-cmd` are guarded by `[ ! -f … ]`, so a reinstall or
|
||||
version upgrade never overwrites operator edits.
|
||||
- Image tags pinned. `ghcr.io/tribler/tribler` is pinned to `v8.4.2`, the digest `:latest`
|
||||
resolved to at the time of pinning.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Generate a random API key in the hook.** The Maison hook toolbox has no `openssl`;
|
||||
`od -An -N32 -tx1 /dev/urandom` would work, but the key has to be written into *two*
|
||||
files that must agree, and a partially-applied seed leaves the feeder unable to talk to
|
||||
its own core. Given the API is unreachable off the app-private network, the added
|
||||
failure mode buys nothing.
|
||||
- **Put Tribler's API behind AppShield.** The feeder is a machine client; it cannot
|
||||
complete an interactive OIDC redirect.
|
||||
- **Run the feeder as `$PUID:$PGID`.** Writes into MetaCore's root-owned tree fail.
|
||||
|
||||
## Data protection
|
||||
|
||||
No user media is mounted. Tribler's own state (`/root/.Tribler`), its seed config, the
|
||||
feeder's cursor and the TMDB cache all live under `/DATA/AppData/metafeedertribler/`, the
|
||||
unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,908 @@
|
||||
name: metafeederusenet
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Usenet — a STANDALONE, self-hosted Usenet feeder for MetaMesh.
|
||||
#
|
||||
# Unlike MetaFeeder · Prowlarr (which proxies an EXTERNAL Newznab/Torznab
|
||||
# indexer and mints an `nzb-release` (0x1005) cid embedding that indexer's
|
||||
# host), this app runs its OWN header scan (an `nntmux` — newznab-tmux fork —
|
||||
# sidecar) so it can mint a PORTABLE `nzb-posting` (0x1003) cid: a digest over
|
||||
# the article Message-ID set, embedding no indexer host, redeemable by ANY
|
||||
# peer with a plain NNTP provider account. It ADDS a Usenet source; it does
|
||||
# not replace the Prowlarr/Torznab one — a gateway query fans out to both.
|
||||
#
|
||||
# Design record (meta-gateway repo): docs/others/self-hosted-usenet-indexer-study.md
|
||||
# Feeder repo: github.com/worph/meta-feeder-usenet
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the shared **MetaCore** app (`metacore-app:9000`) — where resolved
|
||||
# records + TMDB posters are stored; the shared **MetaShare** app then
|
||||
# announces them to the swarm.
|
||||
# The **MetaGateway** app must list this feeder's URL
|
||||
# (`http://metafeederusenet-feeder:8080`) in its feeders config so it
|
||||
# discovers + queries it (MetaGateway, MetaCore and this feeder share `pcs`).
|
||||
#
|
||||
# Everything is scoped under the `metafeederusenet-*` prefix + the per-stack
|
||||
# `metafeederusenet` network + /DATA/AppData/metafeederusenet/ data root, so it
|
||||
# installs alongside the other Meta* apps with zero collisions.
|
||||
#
|
||||
# amd64 ONLY — `pyrowman/nntmux` ships no arm64 build.
|
||||
#
|
||||
# ⚠ Four silent-failure traps in the `pyrowman/nntmux` image, empirically
|
||||
# verified (see the meta-gateway dev compose header for the full writeup):
|
||||
# 1. `PATH_TO_NZBS` (and every other storage path var) must be ABSENT from
|
||||
# the env, never present-but-blank — that reads as an explicit empty
|
||||
# string, not "unset", and nntmux aborts NZB writing with 0 releases and
|
||||
# no visible error. This compose bind-mounts a COMPLETE, hand-filled
|
||||
# `.env` (below) rather than relying on individual `environment:` keys,
|
||||
# for exactly this reason.
|
||||
# 2. `DB_CONNECTION` must be `mariadb`, not `mysql` — the install migration
|
||||
# loads its schema file by connection name; `mysql` finds no matching
|
||||
# schema and fails partway through a from-scratch migration.
|
||||
# 3. Manticore is a HARD first-boot dependency (unless
|
||||
# `ELASTICSEARCH_ENABLED=true`, which just swaps it for a heavier one) —
|
||||
# `metafeederusenet-manticore` below is not optional.
|
||||
# 4. Seeded usenet groups default to `active=0, backfill=0` — scanning an
|
||||
# inactive group silently does nothing. See the tips below for the SQL
|
||||
# to activate one.
|
||||
# One FIRST-BOOT-ONLY loud bug: the entrypoint runs `config:cache` before
|
||||
# `key:generate`, so every session route 500s until you run
|
||||
# `docker exec metafeederusenet-nntmux php artisan config:clear` once. See tips.
|
||||
#
|
||||
# ⚠ Four more, found the hard way on a live box (2026-07-26) after this stack
|
||||
# came up with SEVEN healthy containers and returned ZERO results for every
|
||||
# query — the feeder answering `"Done"` and nothing else. None of them logs
|
||||
# anything; each one alone is enough to keep the catalog empty forever.
|
||||
#
|
||||
# 5. NEVER set a key in `nntmux.env` unless the value must DIFFER from the
|
||||
# PHP default. `config/nntmux.php` does
|
||||
# `env('NN_MULTIPROCESSING_MAX_CHILD_TIME', 1800)` and Laravel's `env()`
|
||||
# returns STRINGS, so restating the default in the .env handed
|
||||
# `Pool::timeout(float)` a `string("1800")` → TypeError → every Forking
|
||||
# multiprocessing runner died on start. Those runners are exactly what a
|
||||
# scan needs. This is trap #1's type-flavoured twin: a redundant
|
||||
# restatement is not a no-op, it silently changes the value's TYPE. The
|
||||
# key is therefore deliberately ABSENT from the seeded .env below (see the
|
||||
# comment there); with it absent, `gettype()` reports `integer` and the
|
||||
# runners execute. Same rule applies to any other key you are tempted to
|
||||
# "document" by writing its default into the file.
|
||||
# (Upstream record: meta-gateway `dev/docker-compose.feeder-usenet.yml`,
|
||||
# the "CORRECTION (2026-07-26)" block.)
|
||||
# 6. The image runs `frankenphp` and NOTHING ELSE — no cron, no supervisord,
|
||||
# no queue worker. nntmux's own `settings.running=1` therefore starts
|
||||
# nothing: there is no process to read it. Even with valid NNTP creds and
|
||||
# active groups the catalog stays empty permanently. That is why
|
||||
# `metafeederusenet-scanner` below exists — it is the missing loop, run as
|
||||
# a SECOND container off the same image so the web container's
|
||||
# install/populate entrypoint stays untouched.
|
||||
# 7. The nntmux entrypoint RE-SEEDS `settings` and `usenet_groups` on EVERY
|
||||
# container boot. Configuration applied before a restart silently reverts
|
||||
# (observed: `running`→0, `delaytime`→2, every group→`active=0`). Apply
|
||||
# group activation and scheduler settings AFTER the final restart, and
|
||||
# re-check them after any `docker restart` / stack update.
|
||||
# 8. `settings.delaytime` (HOURS, default 2) stalls a backfilled catalog:
|
||||
# `ProcessReleases` refuses to advance a collection until it has aged past
|
||||
# it. That is correct for live tailing (a posting may still be uploading)
|
||||
# and pointless for backfill, where every article is already days old —
|
||||
# and the symptom, collections climbing while releases flatline, is
|
||||
# indistinguishable from a broken pipeline because nothing logs an error.
|
||||
# Because trap #7 re-seeds it back to 2 on every restart, the scanner
|
||||
# loop below ASSERTS `delaytime=0` on each pass **while `backfill=1`** —
|
||||
# the one setting it writes, making the tier self-healing across
|
||||
# restarts. It is scoped to the backfill phase on purpose: with
|
||||
# `backfill=0` (pure forward tailing) the operator's own value stands.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederusenet-db — MariaDB backing nntmux's OWN release catalog. Not
|
||||
# shared with the gateway's meta-core Redis or anything else in the mesh.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-db:
|
||||
image: mariadb:12.3
|
||||
cpu_shares: 70
|
||||
container_name: metafeederusenet-db
|
||||
hostname: metafeederusenet-db
|
||||
restart: unless-stopped
|
||||
# 2g, not the 512m this shipped with: 512m is fine for an empty schema and
|
||||
# far too small once the catalog is real. A header scan of a busy binary
|
||||
# group builds `collections`/`binaries`/`parts` tables in the tens of
|
||||
# millions of rows, and InnoDB starved of buffer pool turns collation into
|
||||
# a disk-thrash that never catches up. Measured on a live box.
|
||||
mem_limit: 2g
|
||||
memswap_limit: 2g
|
||||
environment:
|
||||
MARIADB_DATABASE: nntmux
|
||||
MARIADB_USER: nntmux
|
||||
MARIADB_PASSWORD: "$APP_DEFAULT_PASSWORD"
|
||||
MARIADB_RANDOM_ROOT_PASSWORD: "yes"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/db:/var/lib/mysql
|
||||
healthcheck:
|
||||
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 20
|
||||
start_period: 20s
|
||||
networks:
|
||||
- metafeederusenet
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederusenet-redis — nntmux's OWN Redis. Deliberately separate from the
|
||||
# gateway core's Redis (different failure domain; losing this one is free).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-redis:
|
||||
image: redis:7-alpine
|
||||
cpu_shares: 70
|
||||
container_name: metafeederusenet-redis
|
||||
hostname: metafeederusenet-redis
|
||||
restart: unless-stopped
|
||||
mem_limit: 128m
|
||||
memswap_limit: 128m
|
||||
networks:
|
||||
- metafeederusenet
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederusenet-manticore — RT-index search backend. NOT optional (trap
|
||||
# #3 above) — a hard first-boot dependency of nntmux's own entrypoint, even
|
||||
# though this feeder reads `releases` via SQL directly and never touches it.
|
||||
# Digest-pinned (upstream reuses low-looking version numbers across an old
|
||||
# semver line and a newer CalVer-ish one; the digest is unambiguous).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-manticore:
|
||||
image: manticoresearch/manticore@sha256:032388dc77321e3ac898a14b9a0c3fc22ab8d08cd2582abd6c5b642d46b82181
|
||||
cpu_shares: 50
|
||||
container_name: metafeederusenet-manticore
|
||||
hostname: metafeederusenet-manticore
|
||||
restart: unless-stopped
|
||||
mem_limit: 512m
|
||||
memswap_limit: 512m
|
||||
ulimits:
|
||||
nofile: { soft: 65535, hard: 65535 }
|
||||
networks:
|
||||
- metafeederusenet
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederusenet-nntmux — the header scanner + web UI/API. Exposed DIRECTLY
|
||||
# via Caddy (no AppShield gate): nntmux ships its own admin login
|
||||
# (ADMIN_USER/ADMIN_PASS, seeded below), the same convention this store uses
|
||||
# for every app that brings its own auth (Jellyfin, Nextcloud, Immich, ...).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-nntmux:
|
||||
# Digest-pinned, NOT :latest — a silent tag move could change the
|
||||
# `misc/update/*.php` command surface the "collate via tinker" tip below
|
||||
# depends on. Re-pin deliberately, re-verify the trap list above when you do.
|
||||
image: pyrowman/nntmux@sha256:7fc221f7a806fee3e4a2145159e1a147962edc095282bdc692806968f791fabe
|
||||
cpu_shares: 50
|
||||
container_name: metafeederusenet-nntmux
|
||||
hostname: metafeederusenet-nntmux
|
||||
restart: unless-stopped
|
||||
# The working set scales with scan breadth (group set x backfill depth), an
|
||||
# operator choice (see tips), so this is a ceiling rather than a size: 3g
|
||||
# is what a real corpus needed on a live box — 1536m was reached and
|
||||
# exceeded by a single busy group's header scan. Raise it if you scan wide.
|
||||
mem_limit: 3g
|
||||
memswap_limit: 3g
|
||||
environment:
|
||||
# Shell-script-consumed vars — the entrypoint's own `mkdir -p` step reads
|
||||
# these from the PROCESS environment, not the mounted .env. Keep in
|
||||
# lockstep with the matching keys in nntmux.env below.
|
||||
PATH_TO_NZBS: /app/storage/nzb/
|
||||
COVERS_PATH: /app/storage/covers
|
||||
TEMP_UNRAR_PATH: /app/storage/tmp/unrar/
|
||||
TEMP_UNZIP_PATH: /app/storage/tmp/unzip/
|
||||
ELASTICSEARCH_ENABLED: "false"
|
||||
volumes:
|
||||
# The complete, hand-filled .env (trap #1) — a full-file mount rather
|
||||
# than individual `environment:` keys. Writable: `nntmux:install`
|
||||
# persists a generated APP_KEY back into it on first boot.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/nntmux-env/nntmux.env:/app/.env:rw
|
||||
# The .nzb store — shared with metafeederusenet-feeder below,
|
||||
# read-write here (nntmux writes), read-only there (feeder only reads).
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/nntmux-storage/nzb:/app/storage/nzb:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "-o", "/dev/null", "http://localhost/"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 15
|
||||
start_period: 30s
|
||||
depends_on:
|
||||
metafeederusenet-db:
|
||||
condition: service_healthy
|
||||
metafeederusenet-redis:
|
||||
condition: service_started
|
||||
metafeederusenet-manticore:
|
||||
condition: service_started
|
||||
labels:
|
||||
caddy_0: metafeederusenetnntmux-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederusenetnntmux-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederusenetnntmux-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
networks:
|
||||
- metafeederusenet
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederusenet-scanner — the SCAN LOOP (trap #6). The nntmux image runs
|
||||
# frankenphp and nothing else, so without this container nothing ever calls
|
||||
# the scan/backfill/collate entry points and the catalog stays empty forever,
|
||||
# no matter how correct the credentials, the groups or `settings.running` are.
|
||||
#
|
||||
# It is a second container off the SAME digest-pinned image, sharing the same
|
||||
# `/app/.env` and the same nzb store, rather than a second process inside the
|
||||
# web container — the web container's entrypoint runs install/populate under
|
||||
# `set -e` and must keep it; here it is REPLACED by `entrypoint:` below so
|
||||
# that install/populate does NOT run twice against one database.
|
||||
#
|
||||
# Every knob (interval, binaries on/off, backfill on/off, backfill_qty,
|
||||
# backfill_days, running) is read from nntmux's own `settings` table at
|
||||
# /admin/tmux-edit — deliberately NOT from new env vars here, which would
|
||||
# make two sources of truth for one number.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-scanner:
|
||||
# MUST stay byte-identical to metafeederusenet-nntmux's pin above: the two
|
||||
# containers share one `/app/.env` and one database schema, and the script
|
||||
# below calls `misc/update/*.php` + `Blacklight\` classes by name.
|
||||
image: pyrowman/nntmux@sha256:7fc221f7a806fee3e4a2145159e1a147962edc095282bdc692806968f791fabe
|
||||
cpu_shares: 20
|
||||
container_name: metafeederusenet-scanner
|
||||
hostname: metafeederusenet-scanner
|
||||
restart: unless-stopped
|
||||
# Bypasses the image's own install/populate entrypoint on purpose — see the
|
||||
# block comment above. Only the web container may run it.
|
||||
entrypoint: ["/bin/sh", "/scanner.sh"]
|
||||
# The image ships a HEALTHCHECK that curls its own web server. This
|
||||
# container deliberately runs no web server — its entrypoint IS the scan
|
||||
# loop — so the inherited probe can only ever fail. Observed live: a
|
||||
# scanner actively backfilling and writing NZBs sitting at
|
||||
# `Up 5 minutes (unhealthy)`.
|
||||
#
|
||||
# That is not cosmetic. A permanently-red container that is in fact working
|
||||
# trains the operator to ignore health status, which is precisely how this
|
||||
# stack's empty-catalog failure stayed invisible for so long — the feeder's
|
||||
# own health() reporting `ok` over a zero-row catalog was the same bug from
|
||||
# the other direction. Liveness for this container is its LOG
|
||||
# (`docker logs -f metafeederusenet-scanner`), not a probe; a probe that
|
||||
# cannot be honest is worse than none.
|
||||
healthcheck:
|
||||
disable: true
|
||||
# A backfill pass holds a group's article range in memory; 2g is the
|
||||
# headroom measured alongside nntmux's own 3g on a live box.
|
||||
mem_limit: 2g
|
||||
memswap_limit: 2g
|
||||
environment:
|
||||
# Same shell-script-consumed vars as metafeederusenet-nntmux: the wrapper
|
||||
# scripts read these from the PROCESS environment, not from the mounted
|
||||
# .env (trap #1). Keep the two blocks in lockstep.
|
||||
PATH_TO_NZBS: /app/storage/nzb/
|
||||
COVERS_PATH: /app/storage/covers
|
||||
TEMP_UNRAR_PATH: /app/storage/tmp/unrar/
|
||||
TEMP_UNZIP_PATH: /app/storage/tmp/unzip/
|
||||
ELASTICSEARCH_ENABLED: "false"
|
||||
volumes:
|
||||
# The same .env the web container filled in — one credential set, one
|
||||
# APP_KEY. Read-write only because Laravel's .env handling expects it.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/nntmux-env/nntmux.env:/app/.env:rw
|
||||
# nntmux writes .nzb.gz here; the feeder reads them read-only.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/nntmux-storage/nzb:/app/storage/nzb:rw
|
||||
# The loop itself, provisioned by pre-install-cmd below.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/scanner/scanner.sh:/scanner.sh:ro
|
||||
depends_on:
|
||||
# Not just `started`: the loop's very first action is an artisan call, so
|
||||
# it must not race the web container's install/migrate step.
|
||||
metafeederusenet-nntmux:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- metafeederusenet
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederusenet-feeder — the usenet feeder sidecar (meta-feeder-usenet).
|
||||
# Reads nntmux's release catalog + shared .nzb store; self-publishes
|
||||
# resolved records + seeds the `.nzb` manifest to the shared MetaCore. The
|
||||
# gateway reaches it directly on `pcs`; the human configures it through the
|
||||
# AppShield gate below.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-usenet:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeederusenet-feeder
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM instead
|
||||
# of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeederusenet-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# INFRA-ONLY env (the "config: no env if in webui" rule). Operator
|
||||
# config beyond the two DB/path seeds below — TMDB token / language,
|
||||
# meta-core URL — is read from the persisted config.json (this
|
||||
# feeder's config page behind the AppShield gate). These env vars are
|
||||
# wiring only: the enrichment sidecars, the enrich callback, the peer id.
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
# The peer id stamped into provenance.gateway_peer on every record this
|
||||
# feeder self-publishes — names the GATEWAY this feeder serves, not the
|
||||
# feeder itself (analytics group by it). Without it, falls back to HOSTNAME.
|
||||
META_GATEWAY_PEER_ID: "metagateway"
|
||||
FILENAME_PARSER_URL: "http://metafeederusenet-filename-parser:8080"
|
||||
TMDB_PLUGIN_URL: "http://metafeederusenet-tmdb:8080"
|
||||
META_FEEDER_CALLBACK_URL: "http://metafeederusenet-feeder:8080/enrich/callback"
|
||||
# First-boot SEED only (the persisted config.json wins on every later
|
||||
# restart — change these on the /config page after first boot, not here).
|
||||
NNTMUX_DB_URL: "mysql://nntmux:${APP_DEFAULT_PASSWORD}@metafeederusenet-db:3306/nntmux"
|
||||
NNTMUX_NZB_PATH: "/nntmux-nzb"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config (config.json saved via the config UI) + redb cache.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/feeder-state:/data/meta-feeder:rw
|
||||
# Same volume nntmux writes .nzb.gz into — read-only here.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/nntmux-storage/nzb:/nntmux-nzb:ro
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
depends_on:
|
||||
metafeederusenet-nntmux:
|
||||
condition: service_healthy
|
||||
metafeederusenet-filename-parser:
|
||||
condition: service_started
|
||||
metafeederusenet-tmdb:
|
||||
condition: service_started
|
||||
networks:
|
||||
- metafeederusenet
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bundled enrichment plugins (self-contained). filename-parser per file; tmdb
|
||||
# once per release — tmdb writes posters/backdrops to the shared meta-core's
|
||||
# WebDAV (over pcs) so the ingester seeds them. No healthcheck: these plugin
|
||||
# images don't ship one (curl absent).
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet-filename-parser:
|
||||
image: ghcr.io/worph/metamesh-plugin-filename-parser:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeederusenet-filename-parser
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
hostname: metafeederusenet-filename-parser
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
EMIT_TITLE_TAGS: "true"
|
||||
networks:
|
||||
- metafeederusenet
|
||||
|
||||
metafeederusenet-tmdb:
|
||||
image: ghcr.io/worph/metamesh-plugin-tmdb:1.0.2
|
||||
cpu_shares: 30
|
||||
container_name: metafeederusenet-tmdb
|
||||
mem_limit: 384m
|
||||
memswap_limit: 384m
|
||||
hostname: metafeederusenet-tmdb
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
# Posters land in the shared meta-core (over pcs) so the ingester seeds
|
||||
# them. LOAD-BEARING at metamesh-plugin-tmdb:1.0.0 — do NOT remove (see
|
||||
# the sibling feeder apps' identical note: the published image needs
|
||||
# this, unlike the source build the dev stack runs).
|
||||
WEBDAV_URL: "http://metacore-app:9000/webdav"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederusenet/tmdb-cache:/cache
|
||||
networks:
|
||||
- metafeederusenet
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config UI (nntmux DB URL / nzb path, TMDB
|
||||
# token) behind Yundera OIDC SSO. container_name == subdomain prefix. The
|
||||
# gateway still reaches the feeder directly on `pcs` (this gate is for the
|
||||
# human only); index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederusenet:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederusenet
|
||||
hostname: metafeederusenet
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederusenet-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederusenet-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederusenet-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederusenet-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederusenet-feeder
|
||||
networks:
|
||||
- metafeederusenet
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederusenet:
|
||||
name: metafeederusenet
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederusenet
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**A standalone, self-hosted Usenet feeder for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Usenet runs its own Usenet header scan (a bundled
|
||||
**nntmux** indexer) instead of proxying a third-party Newznab site.
|
||||
Because it finds article Message-IDs itself at scan time, it mints a
|
||||
**portable** posting reference — no indexer host baked in, redeemable by
|
||||
any peer with a plain NNTP provider account. It adds a Usenet source
|
||||
alongside **MetaFeeder · Prowlarr**; it doesn't replace it.
|
||||
|
||||
**Self-contained:** ships its own nntmux scanner (MariaDB + Redis +
|
||||
Manticore) + filename-parser + TMDB enrichment, so each release gets a
|
||||
clean title, poster and plot.
|
||||
|
||||
**You still need a Usenet provider account** (NNTP login) — this app
|
||||
scans headers, it does not supply an account, and nothing at all is
|
||||
indexed until you supply one. Configure it after install (see the
|
||||
install tips). Keep the connection count low: this scanner and your
|
||||
playback share the same provider connection slots.
|
||||
|
||||
**The catalog is not instant.** Once configured, a bundled scanner
|
||||
container runs the header-scan / backfill / collation loop on the
|
||||
interval you set. A first pass over a few busy binary groups pulls
|
||||
tens of GB and runs for hours; a deep backfill is hundreds of GB. The
|
||||
install tips show which knobs bound that.
|
||||
|
||||
**Also requires:** **MetaGateway** — its meta-core stores the resolved
|
||||
records and announces them to the swarm. The gateway must list this
|
||||
feeder in its config.
|
||||
|
||||
fr_fr: |
|
||||
**Un feeder Usenet autonome et auto-hébergé pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Usenet effectue lui-même le scan des en-têtes Usenet (un
|
||||
indexeur **nntmux** embarqué) plutôt que de passer par un site Newznab
|
||||
tiers. Comme il retrouve les Message-ID des articles dès le scan, il
|
||||
génère une référence de posting **portable** — sans hôte d'indexeur
|
||||
intégré, exploitable par n'importe quel pair disposant d'un simple
|
||||
compte NNTP. Il ajoute une source Usenet en plus de **MetaFeeder -
|
||||
Indexer**, sans le remplacer.
|
||||
|
||||
**Autonome :** embarque son scanner nntmux (MariaDB + Redis + Manticore)
|
||||
+ ses plugins filename-parser + TMDB.
|
||||
|
||||
**Un compte fournisseur Usenet (NNTP) reste nécessaire** — cette app
|
||||
scanne les en-têtes, elle ne fournit pas d'accès, et rien n'est indexé
|
||||
tant que vous n'en fournissez pas un. À configurer après installation
|
||||
(voir les conseils d'installation). Gardez un nombre de connexions
|
||||
faible : le scan et la lecture partagent les mêmes créneaux du
|
||||
fournisseur.
|
||||
|
||||
**Le catalogue ne se remplit pas instantanément.** Une fois configuré,
|
||||
un conteneur de scan embarqué exécute la boucle scan / backfill /
|
||||
collation à l'intervalle que vous choisissez. Un premier passage sur
|
||||
quelques gros groupes binaires télécharge des dizaines de Go et dure
|
||||
des heures ; un backfill profond, des centaines de Go.
|
||||
|
||||
**Nécessite aussi :** l'app **MetaGateway** (son meta-core stocke et
|
||||
annonce les enregistrements).
|
||||
|
||||
tagline:
|
||||
en_us: Self-hosted Usenet header scanner (bundled nntmux) for the MetaMesh network
|
||||
fr_fr: Scanner Usenet auto-hébergé (nntmux inclus) pour le réseau MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Usenet
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederUsenet/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederUsenet/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederUsenet/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederUsenet/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederUsenet/thumbnail.png
|
||||
|
||||
store_app_id: metafeederusenet
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (this feeder
|
||||
publishes to its shared meta-core).
|
||||
|
||||
**After installing this app, a Usenet provider account is required**
|
||||
(this scans headers itself — it needs a plain NNTP login):
|
||||
1. Edit `/DATA/AppData/metafeederusenet/nntmux-env/nntmux.env` (via
|
||||
SSH or a file-manager app) and fill in `NNTP_SERVER`, `NNTP_PORT`,
|
||||
`NNTP_USERNAME`, `NNTP_PASSWORD` (use the SAME provider account
|
||||
your MetaShare/meta-share peer is already credentialed against —
|
||||
one account, two consumers). Keep `NNTP_CONNECTIONS` low (1-2):
|
||||
the scanner and playback share the same provider connection slots,
|
||||
so a greedy value here stalls playback mid-stream elsewhere.
|
||||
2. Restart the nntmux container: `docker restart metafeederusenet-nntmux`.
|
||||
3. **First boot only:** nntmux's install sequence caches its config
|
||||
before generating an encryption key, so every page 500s until you
|
||||
run `docker exec metafeederusenet-nntmux php artisan config:clear`
|
||||
once. (`curl -sI https://metafeederusenetnntmux-<your-domain>/` — a
|
||||
`500` means you need this; a `302` redirect to `/login` means
|
||||
you're fine.)
|
||||
4. Log in to the nntmux dashboard at the **MetaFeeder Usenet nntmux**
|
||||
link (`ADMIN_USER=admin`, password = this app's auto-generated
|
||||
default password — check CasaOS's app credentials panel) and
|
||||
change it.
|
||||
|
||||
**⚠ Do steps 5-6 AFTER your LAST restart, and re-check them after
|
||||
every later restart or stack update.** The nntmux entrypoint RE-SEEDS
|
||||
the `settings` and `usenet_groups` tables on EVERY container boot, so
|
||||
anything you configure before a restart silently reverts (observed:
|
||||
`running` back to 0, `delaytime` back to 2, every group back to
|
||||
`active=0`) — with no error anywhere.
|
||||
|
||||
5. **Activate at least one usenet group** — seeded groups start
|
||||
inactive (`active=0`), and scanning an inactive group silently
|
||||
touches nothing at all:
|
||||
```
|
||||
docker exec metafeederusenet-db mariadb -unntmux -p"<password>" nntmux \
|
||||
-e "UPDATE usenet_groups SET active=1, backfill=1 WHERE name IN ('alt.binaries.teevee');"
|
||||
```
|
||||
6. **Start the scheduler** in nntmux's admin UI (**/admin/tmux-edit**)
|
||||
or directly in the `settings` table: set `running=1`, pick your
|
||||
`monitor_delay` (seconds between passes), and turn on `binaries`
|
||||
and/or `backfill` with a `backfill_qty` / `backfill_days` you're
|
||||
happy to pay for. The bundled `metafeederusenet-scanner` container
|
||||
polls these settings and runs the scan/backfill/collate loop for
|
||||
you — the nntmux image itself ships NO cron and NO queue worker, so
|
||||
this container is what actually makes a scan happen. There are
|
||||
deliberately no env vars for these knobs: the `settings` table is
|
||||
the single source of truth.
|
||||
|
||||
**`delaytime` (HOURS, default 2) will stall a backfilled catalog.**
|
||||
nntmux won't turn a collection into a release until it has aged
|
||||
past `delaytime`, which is right for tailing live posts and
|
||||
pointless when backfilling history — and the symptom (collections
|
||||
climbing, releases flat, zero query results) looks exactly like a
|
||||
broken pipeline. **While `backfill=1`, the scanner forces
|
||||
`delaytime=0` for you on every pass** and logs
|
||||
`asserting delaytime=0 for the backfill phase (was N)` when it does
|
||||
— it has to re-assert it because a container restart re-seeds the
|
||||
value back to 2. This is the only setting the loop ever writes.
|
||||
|
||||
So if you want a different steady-state `delaytime`, **set
|
||||
`backfill=0` first**, then set your value — otherwise the loop will
|
||||
keep pulling it back to 0. With `backfill=0` the scanner never
|
||||
touches it and `1`-`2` is the sensible live-tailing value.
|
||||
|
||||
**Budget the first scan.** A first header pass over a few busy
|
||||
binary groups pulls tens of GB and runs for hours; a
|
||||
full-retention backfill is hundreds of GB. `backfill_days` is the
|
||||
knob that bounds it — start small, widen once you see releases.
|
||||
|
||||
7. Open this app's config UI (the dashboard link) to optionally set a
|
||||
TMDB v4 token for posters, then Save.
|
||||
8. In **MetaGateway**, make sure this feeder is listed in its feeders
|
||||
config (`http://metafeederusenet-feeder:8080`) so it gets
|
||||
discovered.
|
||||
|
||||
**Note:** which groups to scan, backfill depth and scan interval are
|
||||
your call as the operator — pick them for your own catalog.
|
||||
|
||||
**Watch the loop:** `docker logs -f metafeederusenet-scanner` prints
|
||||
one line per pass (`header scan` / `backfill direct` / `collate
|
||||
<group>` / `sleeping`). `settings.running=0` shows up there as
|
||||
`idle`.
|
||||
|
||||
**Authentication:** the feeder's config UI is protected by Yundera's
|
||||
Authelia (OIDC); nntmux uses its own login (seeded above).
|
||||
|
||||
pre-install-cmd: |
|
||||
# Seed nntmux's .env (trap #1 — must be a complete file, see header).
|
||||
# NNTP_* are deliberately left blank; fill them in per the install tips.
|
||||
if [ ! -f /DATA/AppData/metafeederusenet/nntmux-env/nntmux.env ]; then
|
||||
cat > /DATA/AppData/metafeederusenet/nntmux-env/nntmux.env <<NNTMUX_ENV
|
||||
DB_CONNECTION=mariadb
|
||||
DB_HOST=metafeederusenet-db
|
||||
DB_PORT=3306
|
||||
DB_SOCKET=
|
||||
DB_USERNAME=nntmux
|
||||
DB_PASSWORD=$APP_DEFAULT_PASSWORD
|
||||
DB_DATABASE=nntmux
|
||||
|
||||
MANTICORESEARCH_HOST=metafeederusenet-manticore
|
||||
MANTICORESEARCH_PORT=9308
|
||||
|
||||
ELASTICSEARCH_HOST=localhost
|
||||
ELASTICSEARCH_PORT=9200
|
||||
ELASTICSEARCH_SCHEME=http
|
||||
ELASTICSEARCH_USER=
|
||||
ELASTICSEARCH_PASS=
|
||||
ELASTICSEARCH_LOGGING=false
|
||||
ELASTICSEARCH_ENABLED=false
|
||||
|
||||
NNTP_COMPRESSED_HEADERS=false
|
||||
USE_ALTERNATE_NNTP_SERVER=false
|
||||
|
||||
NNTP_USERNAME=
|
||||
NNTP_PASSWORD=
|
||||
NNTP_SERVER=
|
||||
NNTP_PORT=563
|
||||
NNTP_CONNECTIONS=1
|
||||
NNTP_SSLENABLED=true
|
||||
NNTP_SOCKET_TIMEOUT=120
|
||||
|
||||
NNTP_USERNAME_A=
|
||||
NNTP_PASSWORD_A=
|
||||
NNTP_SERVER_A=
|
||||
NNTP_PORT_A=
|
||||
NNTP_CONNECTIONS_A=
|
||||
NNTP_SSLENABLED_A=false
|
||||
NNTP_SOCKET_TIMEOUT_A=120
|
||||
|
||||
# NN_MULTIPROCESSING_MAX_CHILD_TIME is deliberately ABSENT (trap #5).
|
||||
# Writing it here, even as its own default of 1800, hands
|
||||
# Pool::timeout(float) a string via Laravel's env(), which returns STRINGS,
|
||||
# and every Forking multiprocessing runner dies with a TypeError.
|
||||
# RULE for this whole file: never set a key unless the value must DIFFER
|
||||
# from the PHP default in config/nntmux.php. Restating a default is not
|
||||
# documentation, it is a type change.
|
||||
|
||||
ADMIN_USER=admin
|
||||
ADMIN_PASS=$APP_DEFAULT_PASSWORD
|
||||
ADMIN_EMAIL=admin@$APP_DOMAIN
|
||||
|
||||
APP_NAME=NNTmux
|
||||
APP_ENV=production
|
||||
APP_DEBUG=false
|
||||
APP_TIMEZONE=UTC
|
||||
APP_URL=https://metafeederusenetnntmux-$APP_DOMAIN
|
||||
|
||||
APP_LOCALE=en
|
||||
APP_FALLBACK_LOCALE=en
|
||||
APP_FAKER_LOCALE=en_US
|
||||
|
||||
APP_MAINTENANCE_DRIVER=file
|
||||
APP_MAINTENANCE_STORE=database
|
||||
|
||||
BCRYPT_ROUNDS=12
|
||||
APP_KEY=
|
||||
LOG_CHANNEL=stack
|
||||
LOG_STACK=single
|
||||
PASSWORD_HASH=argon2id
|
||||
|
||||
BROADCAST_CONNECTION=log
|
||||
CACHE_STORE=file
|
||||
QUEUE_CONNECTION=sync
|
||||
|
||||
REDIS_HOST=metafeederusenet-redis
|
||||
REDIS_PASSWORD=null
|
||||
REDIS_PORT=6379
|
||||
REDIS_CLIENT=phpredis
|
||||
|
||||
SESSION_DRIVER=file
|
||||
SESSION_DOMAIN=null
|
||||
SESSION_SECURE_COOKIE=false
|
||||
SESSION_ENCRYPT=false
|
||||
SESSION_COOKIE=nntmux
|
||||
SESSION_PATH=/
|
||||
|
||||
SCOUT_DRIVER=database
|
||||
SCOUT_QUEUE=true
|
||||
|
||||
ITEMS_PER_PAGE=50
|
||||
ITEMS_PER_COVER_PAGE=25
|
||||
MAX_PAGER_RESULTS=125000
|
||||
ECHOCLI=true
|
||||
RENAME_PAR2=false
|
||||
ADD_PAR2=false
|
||||
RENAME_MUSIC_MEDIAINFO=true
|
||||
CACHE_EXPIRY_SHORT=5
|
||||
CACHE_EXPIRY_MEDIUM=10
|
||||
CACHE_EXPIRY_LONG=15
|
||||
|
||||
SSL_CAFILE=
|
||||
SSL_CAPATH=
|
||||
SSL_VERIFY_PEER=false
|
||||
SSL_VERIFY_HOST=false
|
||||
SSL_ALLOW_SELF_SIGNED=true
|
||||
|
||||
SMARTY_DEBUG=false
|
||||
SMARTY_CACHING=false
|
||||
SMARTY_CACHE_LIFE=120
|
||||
SMARTY_COMPILE_CHECK=false
|
||||
SMARTY_FORCE_COMPILE=false
|
||||
SMARTY_CACHE_DRIVER=file
|
||||
|
||||
TMDB_APIKEY=9a4e16adddcd1e86da19bcaf5ff3c2a3
|
||||
TMDB_CACHE=false
|
||||
TMDB_LOG=false
|
||||
TVDB_APIKEY=ef6fb572-bcee-4d99-9b52-90549ad7553a
|
||||
TVDB_PIN=
|
||||
|
||||
TEMP_UNRAR_PATH=/app/storage/tmp/unrar/
|
||||
TEMP_UNZIP_PATH=/app/storage/tmp/unzip/
|
||||
VIEW_COMPILED_PATH=/app/storage/framework/views
|
||||
PATH_TO_NZBS=/app/storage/nzb/
|
||||
COVERS_PATH=/app/storage/covers
|
||||
|
||||
TELESCOPE_ENABLED=false
|
||||
PURGE_INACTIVE_USERS=false
|
||||
OTP_ENABLED=false
|
||||
CHECK_PASSWORDED_RARS=false
|
||||
DELETE_PASSWORDED_RELEASES=false
|
||||
DELETE_POSSIBLE_PASSWORDED_RELEASES=false
|
||||
EXTRACT_USING_RARINFO=false
|
||||
PRIVATE_PROFILES=true
|
||||
STORE_USER_IP=false
|
||||
FORUM_FRONTEND_ENABLED=false
|
||||
NNTMUX_ENV
|
||||
fi
|
||||
# Seed the scan loop run by metafeederusenet-scanner (trap #6). Unlike
|
||||
# nntmux.env above this is NOT guarded by `[ ! -f ]` and is NOT operator
|
||||
# config: it is shipped code, so a store update must be able to replace a
|
||||
# stale copy. Every tunable it reads lives in nntmux's `settings` table.
|
||||
# Quoted heredoc delimiter ('SCANNER_SH') — the script contains $, \ and
|
||||
# nested quotes that must reach the file verbatim, unexpanded.
|
||||
cat > /DATA/AppData/metafeederusenet/scanner/scanner.sh <<'SCANNER_SH'
|
||||
#!/bin/sh
|
||||
# nntmux scan launcher.
|
||||
#
|
||||
# The pyrowman/nntmux image runs frankenphp and NOTHING else: no cron, no
|
||||
# supervisord, no queue worker. So `settings.running=1` changes nothing on
|
||||
# its own, because nothing ever starts the loop. This is that loop.
|
||||
#
|
||||
# It runs as a SECOND CONTAINER off the same image + the same /app/.env and
|
||||
# nzb volume, rather than as a second process inside the web container, so
|
||||
# the web container's entrypoint (install/populate under `set -e`) is left
|
||||
# untouched.
|
||||
#
|
||||
# INVOCATION IS NOT FREE CHOICE. Measured PER RUNNER on a live box, NOT a
|
||||
# blanket "Forking works":
|
||||
# * multiprocessing/binaries.php — Forking OK.
|
||||
# * multiprocessing/backfill.php — NO-OP. Reports "...3 job(s) to do
|
||||
# using a max of 0 child process(es)" and returns in 0s having done
|
||||
# nothing. Backfill must be driven DIRECTLY, per group (see the
|
||||
# backfill block below).
|
||||
# * collation MUST go through tinker/ProcessReleases DIRECTLY.
|
||||
# multiprocessing/releases.php aborts every batch on a benign PHP
|
||||
# warning (ReleaseCleaning.php:165) that turns fatal inside the Spatie
|
||||
# child -> 0 releases while collections pile into the tens of
|
||||
# thousands.
|
||||
#
|
||||
# Every knob comes from nntmux's own `settings` table (edited at
|
||||
# /admin/tmux-edit), never from env — otherwise there are two sources of
|
||||
# truth for one number.
|
||||
set -u
|
||||
|
||||
cd /app || exit 1
|
||||
|
||||
setting() { php artisan tinker --execute="echo DB::table('settings')->where('name','$1')->value('value');" 2>/dev/null | tr -dc '0-9'; }
|
||||
set_setting() { php artisan tinker --execute="DB::table('settings')->where('name','$1')->update(['value'=>'$2']);" >/dev/null 2>&1; }
|
||||
groups() { php artisan tinker --execute="foreach (DB::table('usenet_groups')->where('active',1)->pluck('name') as \$g) echo \$g, PHP_EOL;" 2>/dev/null | grep -E '^[a-z0-9.]+$'; }
|
||||
|
||||
log() { echo "[scanner $(date -u +%H:%M:%S)] $*"; }
|
||||
|
||||
log "launcher started"
|
||||
while true; do
|
||||
if [ "$(setting running)" != "1" ]; then
|
||||
log "settings.running=0 — idle"
|
||||
sleep 30
|
||||
continue
|
||||
fi
|
||||
|
||||
if [ "$(setting binaries)" = "1" ]; then
|
||||
qty="$(setting backfill_qty)"; [ -n "$qty" ] || qty=20000
|
||||
log "header scan (qty=$qty)"
|
||||
php misc/update/multiprocessing/binaries.php "$qty" 2>&1 | tail -3
|
||||
fi
|
||||
|
||||
# Backfill must run DIRECT, per group. multiprocessing/backfill.php
|
||||
# reports "max of 0 child process(es)" and silently does nothing — the
|
||||
# Forking wrapper leaves maxProcesses=0 for this runner specifically.
|
||||
# Depth is capped by settings.backfill_days, so this reaches back that
|
||||
# far and then stops advancing; it is NOT a full-retention backfill.
|
||||
if [ "$(setting backfill)" = "1" ]; then
|
||||
# The ONLY setting this loop writes. `delaytime` (hours, default 2)
|
||||
# blocks ProcessReleases from advancing a collection until it ages
|
||||
# past it. Correct for the leading edge — a posting may still be
|
||||
# uploading — but pointless for backfill, where every article is
|
||||
# already days old. Left at 2 during backfill it stalls the catalog
|
||||
# for two hours with a failure mode indistinguishable from a broken
|
||||
# pipeline (collections climb, releases flatline, nothing logs an
|
||||
# error).
|
||||
#
|
||||
# Asserted every pass, not once at boot, because the nntmux
|
||||
# entrypoint RE-SEEDS `settings` on every container start (trap #7)
|
||||
# — a restart silently puts this back to 2 and the tier quietly
|
||||
# stops producing. Scoped to `backfill=1` so a pure forward-tailing
|
||||
# deployment (`backfill=0`) keeps the operator's own value: this is
|
||||
# a per-phase mode, not a preference we get to override.
|
||||
if [ "$(setting delaytime)" != "0" ]; then
|
||||
log "asserting delaytime=0 for the backfill phase (was $(setting delaytime))"
|
||||
set_setting delaytime 0
|
||||
fi
|
||||
qty="$(setting backfill_qty)"; [ -n "$qty" ] || qty=20000
|
||||
log "backfill direct (backfill_days=$(setting backfill_days), qty=$qty)"
|
||||
groups | while read -r g; do
|
||||
[ -n "$g" ] || continue
|
||||
php misc/update/backfill.php "$g" "$qty" 2>&1 | grep -iE "articles|backfill|Group|no new" | tail -2
|
||||
done
|
||||
fi
|
||||
|
||||
# Collation, per active group, DIRECT (never multiprocessing/releases.php).
|
||||
# Never wrap this in a short `timeout`: killing it between the release
|
||||
# INSERT and createNZBs() leaves a release row with no .nzb.gz, which
|
||||
# the feeder then correctly skips forever.
|
||||
groups | while read -r g; do
|
||||
[ -n "$g" ] || continue
|
||||
log "collate $g"
|
||||
php artisan tinker --execute="(new Blacklight\\processing\\ProcessReleases)->processReleases(1, 1, \"$g\", new Blacklight\\NNTP);" 2>&1 | grep -iE "releases (ready|created)|Added [0-9]+ releases|NZB" | tail -3
|
||||
done
|
||||
|
||||
delay="$(setting monitor_delay)"; [ -n "$delay" ] || delay=30
|
||||
log "pass complete; sleeping ${delay}s"
|
||||
sleep "$delay"
|
||||
done
|
||||
SCANNER_SH
|
||||
chmod 755 /DATA/AppData/metafeederusenet/scanner/scanner.sh
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederusenet/db
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeederusenet/nntmux-env
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeederusenet/nntmux-storage/nzb
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeederusenet/feeder-state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeederusenet/tmdb-cache
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafeederusenet/scanner
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederusenet-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,64 @@
|
||||
# MetaFeeder · Usenet — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metafeederusenet-feeder` runs `user: 0:0`.
|
||||
2. The **NNTmux web UI is published on its own hostname**
|
||||
(`metafeederusenetnntmux-<user>.<domain>`) with no AppShield sidecar in front of it — it
|
||||
is protected by NNTmux's own login only.
|
||||
3. The seeded `nntmux.env` contains TMDB and TVDB API keys as literals.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. The feeder publishes resolved records to the shared MetaCore and writes plugin artwork
|
||||
through its WebDAV endpoint; those targets are created and owned by MetaCore's
|
||||
root-running container.
|
||||
2. NNTmux is a Laravel application with its own user table, session handling and admin
|
||||
role. Its admin account is created at install from `$APP_DEFAULT_PASSWORD`, so
|
||||
authentication is **enabled by default** — the "app's own built-in auth" alternative the
|
||||
security checklist accepts. Fronting it with AppShield as well would double-prompt on
|
||||
every page and breaks its own OTP and password-reset flows.
|
||||
3. Those two keys are NNTmux's upstream project defaults, shipped in its own
|
||||
`.env.example`. They are shared read-only application keys for public metadata catalogs,
|
||||
not user credentials, and they are what makes the app work immediately after
|
||||
installation with no manual configuration. An operator can replace them from the NNTmux
|
||||
admin UI.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- The NNTmux admin password is `$APP_DEFAULT_PASSWORD` — the platform-generated secret —
|
||||
and the database password comes from the same variable. Neither is a hardcoded literal,
|
||||
and both are surfaced to the user in `tips.before_install`.
|
||||
- The feeder's own configuration UI *is* behind AppShield + Authelia SSO
|
||||
(`ghcr.io/yundera/appshield:2.0.9`).
|
||||
- `metafeederusenet-db`, `-redis` and `-manticore` are on the app-private
|
||||
`metafeederusenet` bridge network, carry no Caddy labels and publish no host port, so no
|
||||
sibling app on the shared `pcs` network can resolve or dial them.
|
||||
- NNTP provider credentials are deliberately left **blank** in the seed; the user fills
|
||||
them in per `tips.before_install`. Nothing ships with a working news account.
|
||||
- The `nntmux.env` seed is guarded by `[ ! -f … ]`, so a reinstall or version upgrade never
|
||||
overwrites the operator's provider settings. `scanner.sh` is intentionally rewritten on
|
||||
every run — it is shipped code, not operator config, so a store update must be able to
|
||||
replace a stale copy; the write is idempotent.
|
||||
- No user directories are mounted. All binds are under `/DATA/AppData/metafeederusenet/`,
|
||||
declared in `x-compose-app.folders`.
|
||||
- `cpu_shares` on all nine services; every image pinned (the NNTmux and Manticore images by
|
||||
digest).
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **AppShield in front of NNTmux.** Two independent login gates on one UI; its OTP,
|
||||
password-reset and API endpoints break behind the OIDC redirect.
|
||||
- **Blank the TMDB/TVDB keys.** The app then installs into a state where covers and
|
||||
metadata silently fail until the user finds two upstream registration flows — a direct
|
||||
violation of "works immediately after installation, no manual configuration required for
|
||||
basic functionality".
|
||||
- **Run the feeder as `$PUID:$PGID`.** Writes into MetaCore's root-owned tree fail.
|
||||
|
||||
## Data protection
|
||||
|
||||
All state — MariaDB, the NZB store, the feeder cursor, the TMDB cache — is under
|
||||
`/DATA/AppData/metafeederusenet/`, the unit Maison archives on uninstall and restores from
|
||||
backup, so an uninstall/reinstall with "keep user data" comes back with the release
|
||||
catalogue intact rather than re-scanning Usenet from scratch. No user media directory is
|
||||
mounted into any service.
|
||||
|
After Width: | Height: | Size: 165 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 50 KiB |
|
After Width: | Height: | Size: 130 KiB |
@@ -0,0 +1,157 @@
|
||||
name: metafeederwikicommons
|
||||
|
||||
# =============================================================================
|
||||
# MetaFeeder · Wikimedia Commons — open-licensed media for MetaMesh.
|
||||
#
|
||||
# Keyless: finds open-licensed images on Wikimedia Commons and returns the
|
||||
# records to the MetaGateway core, which stores + announces them. Split out of
|
||||
# the former MetaFeeder · Commons app (one provider per app).
|
||||
#
|
||||
# It needs, over the shared `pcs` network:
|
||||
# - the **MetaGateway** app — which must list this feeder's URL
|
||||
# (`http://metafeederwikicommons-feeder:8080`) in its feeders config so it gets discovered.
|
||||
# Discovery is startup-only, so restart the gateway afterwards.
|
||||
#
|
||||
# Everything is scoped under the `metafeederwikicommons-*` prefix + the per-stack `metafeederwikicommons` network +
|
||||
# /DATA/AppData/metafeederwikicommons/ data root, so it installs beside the other Meta* apps with
|
||||
# zero collisions.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metafeederwikicommons-feeder — the wikicommons feeder. Keyless; no config needed.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederwikicommons-feeder:
|
||||
image: ghcr.io/worph/meta-feeder-wikicommons:1.0.0
|
||||
cpu_shares: 30
|
||||
container_name: metafeederwikicommons-feeder
|
||||
hostname: metafeederwikicommons-feeder
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 8080
|
||||
environment:
|
||||
RUST_LOG: "info,meta_feeder=info"
|
||||
META_FEEDER_STATE_DIR: "/data/meta-feeder"
|
||||
TZ: "$TZ"
|
||||
volumes:
|
||||
# Persist feeder config.json + redb cache so it survives recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafeederwikicommons/state:/data/meta-feeder:rw
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metafeederwikicommons
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the feeder's config page behind Yundera OIDC SSO.
|
||||
# container_name == subdomain prefix (the auth-registrar derives the client_id
|
||||
# from it). The gateway still reaches the feeder directly on `pcs`; index → /config.
|
||||
# ---------------------------------------------------------------------------
|
||||
metafeederwikicommons:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafeederwikicommons
|
||||
hostname: metafeederwikicommons
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metafeederwikicommons-feeder"
|
||||
BACKEND_PORT: "8080"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metafeederwikicommons-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metafeederwikicommons-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metafeederwikicommons-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metafeederwikicommons-feeder
|
||||
networks:
|
||||
- metafeederwikicommons
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metafeederwikicommons:
|
||||
name: metafeederwikicommons
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metafeederwikicommons
|
||||
webui_port: 80
|
||||
index: /config
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Wikimedia Commons for the MetaMesh network.**
|
||||
|
||||
MetaFeeder · Wikimedia Commons bridges open-licensed media from **Wikimedia
|
||||
Commons** into MetaMesh. Keyless — no setup required.
|
||||
**Requires:** **MetaGateway** — its meta-core stores the resolved records and
|
||||
announces them to the swarm. The gateway must list this feeder in its config.
|
||||
fr_fr: |
|
||||
**Wikimedia Commons pour le réseau MetaMesh.**
|
||||
|
||||
MetaFeeder · Wikimedia Commons connecte les médias sous licence libre de
|
||||
**Wikimedia Commons** à MetaMesh. Sans clé ni configuration. **Nécessite**
|
||||
l'app **MetaGateway**.
|
||||
tagline:
|
||||
en_us: Wikimedia Commons open-licensed media for MetaMesh
|
||||
fr_fr: Médias libres Wikimedia Commons pour MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaFeeder · Wikimedia Commons
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederWikiCommons/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFeederWikiCommons/thumbnail.png
|
||||
|
||||
store_app_id: metafeederwikicommons
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Install order:** install **MetaGateway** first (its meta-core stores and
|
||||
announces what this feeder finds).
|
||||
|
||||
**After installing this app:** in **MetaGateway**, add this feeder from the
|
||||
dashboard's *Add a feeder* box — name `wikicommons-feeder`, URL `http://metafeederwikicommons-feeder:8080`.
|
||||
The gateway restarts itself: feeder discovery is startup-only.
|
||||
No other setup.
|
||||
**Authentication:** the config UI is protected by Yundera's Authelia (OIDC).
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metafeederwikicommons/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafeederwikicommons-${domain}
|
||||
webui-path: "/config"
|
||||
|
After Width: | Height: | Size: 134 KiB |
@@ -0,0 +1,33 @@
|
||||
# MetaFeeder · Wikimedia Commons — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
`metafeederwikicommons-feeder` runs `user: 0:0`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
Carried over unchanged from the listing this app was split out of (MetaFeeder · Commons), so the
|
||||
feeder behaves exactly as it did there. The MetaFeeder images write their state and
|
||||
redb cache as the container user and some reach MetaCore's root-owned tree; a uid-1000
|
||||
process is refused.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The feeder carries no
|
||||
Caddy labels and publishes no host port, so it is reachable only from the `pcs` Docker
|
||||
network.
|
||||
- The container mounts **no user directories** — not `/DATA/Documents`, `/DATA/Downloads`,
|
||||
`/DATA/Media` or `/DATA/Gallery`. Its only bind is its own state under
|
||||
`/DATA/AppData/metafeederwikicommons/`, declared in `x-compose-app.folders` and owned `$PUID:$PGID`.
|
||||
- `cpu_shares` set on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run as `$PUID:$PGID`.** Not validated for this image; switching the runtime user in
|
||||
the same change as the split would mix a behaviour change into a packaging change.
|
||||
|
||||
## Data protection
|
||||
|
||||
The feeder holds no user media. Its entire state lives under `/DATA/AppData/metafeederwikicommons/`, which
|
||||
is the unit Maison archives on uninstall and restores from backup.
|
||||
|
After Width: | Height: | Size: 134 KiB |
@@ -0,0 +1,215 @@
|
||||
name: metafuse
|
||||
|
||||
services:
|
||||
# Backend service. Reachable on the pcs network as `metafuse-app`.
|
||||
# Public ingress goes through the `metafuse` hash-lock proxy below.
|
||||
metafuse-app:
|
||||
image: ghcr.io/worph/meta-fuse:1.1.0
|
||||
cpu_shares: 50
|
||||
container_name: metafuse-app
|
||||
hostname: metafuse-app
|
||||
restart: unless-stopped
|
||||
privileged: true
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
# FUSE capabilities required for virtual filesystem
|
||||
devices:
|
||||
- /dev/fuse:/dev/fuse
|
||||
|
||||
cap_add:
|
||||
- SYS_ADMIN
|
||||
- DAC_READ_SEARCH
|
||||
- DAC_OVERRIDE
|
||||
|
||||
security_opt:
|
||||
- apparmor:unconfined
|
||||
|
||||
volumes:
|
||||
# Shared infrastructure state (leader election, Redis dump, service discovery)
|
||||
# Owned by MetaCore; mounted rw here for service registration + leader reads.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metacore/meta-core:/meta-core:rw
|
||||
|
||||
# FUSE virtual filesystem mount point
|
||||
- ${DATA_ROOT:-/DATA}/MetaFuse/Library:/mnt/virtual:shared
|
||||
|
||||
# Configuration storage
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metafuse/config:/meta-fuse/config:rw
|
||||
|
||||
# Note: no /files mounts — files are accessed via WebDAV from MetaCore
|
||||
# (Architecture V3). MetaCore is the single source of truth for media files.
|
||||
|
||||
environment:
|
||||
- NODE_ENV=production
|
||||
- META_CORE_PATH=/meta-core
|
||||
- CONFIG_DIR=/meta-fuse/config
|
||||
- SERVICE_NAME=meta-fuse
|
||||
- SERVICE_VERSION=1.0.0
|
||||
- BASE_URL=https://metafuse-$APP_DOMAIN
|
||||
- TZ=$TZ
|
||||
|
||||
# Note: Runs as root for FUSE mount operations
|
||||
user: 0:0
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
# OIDC perimeter for the MetaFuse dashboard. Self-registers as an OIDC client
|
||||
# with the platform's auth-registrar on first login (no secrets to manage).
|
||||
#
|
||||
# Naming: container_name MUST equal the URL-prefix of the public hostname so
|
||||
# the registrar's redirect-URI allowlist accepts `metafuse-<user>.<domain>`.
|
||||
metafuse:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metafuse
|
||||
hostname: metafuse
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
BACKEND_HOST: "metafuse-app"
|
||||
BACKEND_PORT: "80"
|
||||
LISTEN_PORT: "80"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
|
||||
depends_on:
|
||||
- metafuse-app
|
||||
|
||||
labels:
|
||||
# ---- metafuse-${APP_DOMAIN} (gateway TLS) ----
|
||||
caddy_0: metafuse-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
# Unauthenticated paths bypass the hash-lock proxy and hit the backend directly.
|
||||
caddy_0.0_handle: "/health"
|
||||
caddy_0.0_handle.reverse_proxy: "metafuse-app:80"
|
||||
caddy_0.1_handle: "/webdav*"
|
||||
caddy_0.1_handle.reverse_proxy: "metafuse-app:80"
|
||||
caddy_0.2_handle: ""
|
||||
caddy_0.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metafuse-IP.nip.io (gateway TLS) ----
|
||||
caddy_1: metafuse-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.0_handle: "/health"
|
||||
caddy_1.0_handle.reverse_proxy: "metafuse-app:80"
|
||||
caddy_1.1_handle: "/webdav*"
|
||||
caddy_1.1_handle.reverse_proxy: "metafuse-app:80"
|
||||
caddy_1.2_handle: ""
|
||||
caddy_1.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metafuse-IP.sslip.io (Let's Encrypt) ----
|
||||
caddy_2: metafuse-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.0_handle: "/health"
|
||||
caddy_2.0_handle.reverse_proxy: "metafuse-app:80"
|
||||
caddy_2.1_handle: "/webdav*"
|
||||
caddy_2.1_handle.reverse_proxy: "metafuse-app:80"
|
||||
caddy_2.2_handle: ""
|
||||
caddy_2.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Access your media library through a beautifully organized virtual filesystem.**
|
||||
|
||||
MetaFuse creates a virtual view of your media (Movies, TV Shows, Anime) organized by metadata - while your actual files stay exactly where they are. Access via FUSE mount or WebDAV network share.
|
||||
|
||||
**Key Features:**
|
||||
- **Zero File Movement** - Your files stay in place, MetaFuse creates an organized virtual view
|
||||
- **FUSE Virtual Filesystem** - Mount as a local directory on your system
|
||||
- **WebDAV Network Access** - Access your organized library as a network drive from any device
|
||||
- **Smart Organization** - Files organized by title, year, season, episode automatically
|
||||
- **Real-time Updates** - Virtual filesystem updates instantly when MetaSort processes new files
|
||||
- **Plex/Jellyfin Ready** - Point your media server to the organized virtual library
|
||||
|
||||
**Requires:** MetaCore (install first)
|
||||
|
||||
**Perfect for:** Plex/Jellyfin users, media archivists, or anyone wanting an organized view without reorganizing files
|
||||
|
||||
fr_fr: |
|
||||
**Accedez a votre bibliotheque multimedia via un systeme de fichiers virtuel magnifiquement organise.**
|
||||
|
||||
MetaFuse cree une vue virtuelle de vos medias (Films, Series TV, Anime) organisee par metadonnees - pendant que vos fichiers restent exactement ou ils sont. Acces via montage FUSE ou partage reseau WebDAV.
|
||||
|
||||
**Fonctionnalites Cles:**
|
||||
- **Zero Deplacement de Fichiers** - Vos fichiers restent en place, MetaFuse cree une vue virtuelle organisee
|
||||
- **Systeme de Fichiers Virtuel FUSE** - Montez comme un repertoire local sur votre systeme
|
||||
- **Acces Reseau WebDAV** - Accedez a votre bibliotheque organisee comme un lecteur reseau
|
||||
- **Organisation Intelligente** - Fichiers organises par titre, annee, saison, episode automatiquement
|
||||
- **Mises a Jour Temps Reel** - Le systeme virtuel se met a jour instantanement
|
||||
|
||||
**Necessite:** MetaCore (installer en premier)
|
||||
|
||||
tagline:
|
||||
en_us: Virtual filesystem for organized media access - FUSE mount and WebDAV without moving files
|
||||
fr_fr: Systeme de fichiers virtuel pour acces media organise - Montage FUSE et WebDAV sans deplacer de fichiers
|
||||
|
||||
title:
|
||||
en_us: MetaFuse
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFuse/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFuse/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFuse/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFuse/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaFuse/thumbnail.png
|
||||
|
||||
index: /
|
||||
store_app_id: metafuse
|
||||
main: metafuse
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Requires MetaCore** - Install MetaCore first.
|
||||
|
||||
MetaFuse connects to MetaCore for metadata. The dashboard is protected by
|
||||
Yundera's built-in Authelia via OIDC — sign in with your existing Yundera account.
|
||||
|
||||
**FUSE Mount:** Your organized virtual library is exposed on the host at
|
||||
`/DATA/MetaFuse/Library/` — deliberately outside `/DATA/AppData/` so you and other
|
||||
apps (Jellyfin, Plex, Samba, …) can browse it. It is a *virtual* view of records
|
||||
MetaCore already holds, not a second copy of your files. See `rationale.md`.
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metacore/meta-core
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metafuse/config
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/MetaFuse/Library
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metafuse-${domain}
|
||||
webui-path: "/"
|
||||
|
After Width: | Height: | Size: 128 KiB |
@@ -0,0 +1,59 @@
|
||||
# MetaFuse — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metafuse-app` runs `privileged: true`, `user: 0:0`, with `/dev/fuse`,
|
||||
`cap_add: [SYS_ADMIN, DAC_READ_SEARCH, DAC_OVERRIDE]` and `apparmor:unconfined`.
|
||||
2. It mounts `/DATA/MetaFuse/Library`, which is outside both
|
||||
`/DATA/AppData/metafuse/` and the standard user directories, with `shared` mount
|
||||
propagation.
|
||||
3. It mounts `/DATA/AppData/metacore/meta-core` — another app's AppData directory.
|
||||
4. `/webdav*` bypasses the SSO gate.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. MetaFuse *is* a FUSE filesystem. Mounting one from inside a container requires
|
||||
`/dev/fuse`, `SYS_ADMIN`, and an unconfined AppArmor profile; the `DAC_*` caps let the
|
||||
filesystem answer `getattr`/`open` for files owned by other uids. There is no
|
||||
unprivileged path to this feature.
|
||||
2. `shared` propagation is what makes the mount visible on the **host** at
|
||||
`/DATA/MetaFuse/Library` instead of dying inside the container's namespace. That host
|
||||
path is the entire point of the app: it is the directory the user (and Jellyfin, Plex,
|
||||
Samba, …) browse. It is deliberately *not* under `/DATA/AppData/` because it is
|
||||
user-facing content, not application state.
|
||||
3. `/meta-core` is MetaCore's service-discovery and leader-election directory. MetaMesh
|
||||
apps coordinate through it; MetaFuse writes its own service registration there and
|
||||
reads the current leader. MetaCore is a hard prerequisite of this app.
|
||||
4. WebDAV clients (media servers, file managers) authenticate at the WebDAV layer and
|
||||
cannot complete an interactive OIDC redirect.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- The interactive dashboard is behind AppShield + Authelia SSO
|
||||
(`ghcr.io/yundera/appshield:2.0.9`). Only `/health` and `/webdav*` are exempt.
|
||||
- `/DATA/MetaFuse/Library` is a **virtual** view: it projects records already held by
|
||||
MetaCore. It is not a second copy of the user's library and holds no independent data.
|
||||
- Everything MetaFuse persists is under `/DATA/AppData/metafuse/config/`, declared in
|
||||
`x-compose-app.folders`.
|
||||
- Image tags pinned; `cpu_shares` set on both services.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Unprivileged FUSE (`fusermount` with `user_allow_other`).** The container still needs
|
||||
`/dev/fuse` and `SYS_ADMIN` to create the mount namespace; dropping `privileged` left
|
||||
the mount failing at startup.
|
||||
- **Mount under `/DATA/AppData/metafuse/Library`.** It would work, but the resulting path
|
||||
is not somewhere a user is expected to browse, and other apps' file pickers hide
|
||||
`AppData`. The whole feature is "a library folder you can open".
|
||||
- **Put `/webdav*` behind the SSO gate.** Every non-browser WebDAV client then fails to
|
||||
authenticate, which removes the reason the endpoint exists.
|
||||
- **Bundle a private MetaCore.** Duplicates the Redis/leader tier per app and breaks the
|
||||
single-source-of-truth model the ecosystem is built on.
|
||||
|
||||
## Data protection
|
||||
|
||||
MetaFuse stores no user media of its own — the library it exposes is a projection of
|
||||
MetaCore's records. Its own configuration lives in `/DATA/AppData/metafuse/config/`, inside
|
||||
the folder Maison archives and restores, so uninstall/reinstall preserves it. The shared
|
||||
`/meta-core` mount is read-mostly: MetaFuse writes only its own service-registration file
|
||||
under `/meta-core/services/`.
|
||||
|
After Width: | Height: | Size: 105 KiB |
|
After Width: | Height: | Size: 105 KiB |
|
After Width: | Height: | Size: 105 KiB |
|
After Width: | Height: | Size: 128 KiB |
@@ -0,0 +1,110 @@
|
||||
# MetaGateway
|
||||
|
||||
Source-bridging tier for MetaMesh: ingests content from external sources and
|
||||
makes it discoverable across the federation, so **MetaWatch** (and other
|
||||
MetaShare peers) can find and stream it. Unifies the three dev gateway
|
||||
sub-stacks (`docker-compose.yml` + `.torznab.yml` + `.tribler.yml`) into one
|
||||
app sharing a single meta-core, ingester and enrichment plugin pair.
|
||||
|
||||
## What's in the stack (13 containers)
|
||||
|
||||
| Container | Role |
|
||||
|-----------|------|
|
||||
| `metagateway-app` | Gateway peer (feeders-only); writes resolved records to meta-core (dashboard behind OIDC) |
|
||||
| `metagateway-core` | Dedicated meta-core; auto-store target + WebDAV poster store (watcher ON) |
|
||||
| `metagateway-share` | meta-share **ingester**: announces resolved records on the swarm, seeds blobs |
|
||||
| `metagateway-prowlarr` | Bundled Prowlarr torznab aggregator (own public route + forms login) |
|
||||
| `metagateway-torznab-feeder` | torznab feeder → Prowlarr (+ librqbit BT fetch) |
|
||||
| `metagateway-tribler` | Headless Tribler core (IPv8 overlay) |
|
||||
| `metagateway-tribler-feeder` | tribler feeder → Tribler core |
|
||||
| `book-feeder` / `paper-feeder` / `common-feeder` | Gutenberg / arXiv+PubMed / Wikimedia Commons |
|
||||
| `metagateway-filename-parser` / `metagateway-tmdb` | Reused meta-sort enrichment plugins |
|
||||
| `metagateway` | nginx-hash-lock OIDC perimeter for the dashboard |
|
||||
|
||||
The gateway discovers its feeders from `gateway-config.json` (the `feeders`
|
||||
map), and `gateway-enabled.json` opts the `torznab` + `tribler` upstreams in.
|
||||
Both are written by `pre-install-cmd` into `/DATA/AppData/metagateway/state/` —
|
||||
edit them there and restart `metagateway-app` to change the feeder set.
|
||||
|
||||
## Self-contained meta-core
|
||||
|
||||
The gateway's meta-core is **private to the gateway** (it holds resolved feeder
|
||||
records, not your media library), so this app does **not** use the shared
|
||||
MetaCore app and coexists with MetaWatch + the standalone Meta* apps without
|
||||
container-name or data-path collisions.
|
||||
|
||||
```
|
||||
/DATA/AppData/metagateway/
|
||||
├── metacore/ # gateway meta-core (shared by core + ingester)
|
||||
├── files/ # WebDAV file store (tmdb posters/backdrops, seeded blobs)
|
||||
├── state/ # gateway-config.json + gateway-enabled.json + redb caches
|
||||
├── prowlarr/ # Prowlarr config + db
|
||||
├── tribler/ # Tribler IPv8 identity + metadata db (+ installed config)
|
||||
└── tmdb-cache/ # tmdb plugin response cache
|
||||
```
|
||||
|
||||
## First-run setup
|
||||
|
||||
1. **Prowlarr** — open `https://metagateway-prowlarr-<domain>/`, finish the
|
||||
wizard, add indexers, copy the **API Key**.
|
||||
2. Set `PROWLARR_API_KEY` and `TMDB_TOKEN` (TMDB v4 read token) on this app,
|
||||
then restart. Until set, the **torznab feeder reports as degraded** (the
|
||||
reason is visible at its `/health` and in the dashboard) but never crashes —
|
||||
the rest of the gateway stays healthy and the app boots normally. Optional:
|
||||
`GIPHY_API_KEY` (common-feeder), `SCIHUB_MIRRORS` (paper-feeder).
|
||||
3. **Tribler** needs no setup — the baked config installs on first boot.
|
||||
|
||||
## Federation
|
||||
|
||||
Two peers join the **public** libp2p swarm, on two different ports:
|
||||
|
||||
| Peer | Port | Role |
|
||||
|------|------|------|
|
||||
| `metagateway-app` | TCP **4002** | gateway **discovery** — publishes the kad provider record under the `metamesh-gateway` namespace; this is the peer a remote meta-search finds and dials for live fan-out |
|
||||
| `metagateway-share` | TCP **4005** | the ingester's records + bitswap byte-serving (cohort `metamesh-share-default`) |
|
||||
|
||||
Tribler's IPv8 overlay is UDP **8092**.
|
||||
|
||||
**Cross-node discovery needs both the port open inbound AND a public address
|
||||
set.** Without `META_GATEWAY_PUBLIC_ADDR` the gateway starts in local/mDNS-only
|
||||
mode: it pulls in no kad bootstraps, publishes no provider record, and binds an
|
||||
ephemeral port — so no off-host MetaWatch can ever find it, however open the
|
||||
firewall is. The compose sets it from `${APP_PUBLIC_IPV4}`; on a PCS whose
|
||||
public IP is not the v4 one, override it. Same-host MetaWatch discovery works
|
||||
over mDNS on `pcs` regardless.
|
||||
|
||||
## Images
|
||||
|
||||
| Image | Tag |
|
||||
|-------|-----|
|
||||
| `ghcr.io/worph/meta-gateway` | `1.0.25` |
|
||||
| `ghcr.io/worph/meta-share` | `1.0.26` |
|
||||
| `ghcr.io/worph/meta-core` | `1.0.12` |
|
||||
| `ghcr.io/yundera/appshield` | `2.0.9` |
|
||||
|
||||
No feeder images are listed here any more: every content source is a **separate
|
||||
store app**, one per provider (MetaFeeder · Gutenberg / arXiv / Europe PMC /
|
||||
Wikimedia Commons / Prowlarr / Tribler / …), each
|
||||
pinning its own images. The gateway discovers them by URL over `pcs` and
|
||||
soft-skips any that aren't installed.
|
||||
| `lscr.io/linuxserver/prowlarr` | `latest` |
|
||||
| `ghcr.io/yundera/nginx-hash-lock` | `1.0.7` |
|
||||
|
||||
**Tribler** uses the upstream `ghcr.io/tribler/tribler` image directly (there is
|
||||
no custom `meta-tribler` image). `pre-install-cmd` fetches the complete REST
|
||||
config (bound to `0.0.0.0:8085`, key `changeme`) to
|
||||
`/DATA/AppData/metagateway/tribler-config/configuration.json` and bind-mounts it
|
||||
at `/seed/configuration.json`; the entrypoint installs it into `/root/.Tribler`
|
||||
on first boot.
|
||||
|
||||
## Resilience notes
|
||||
|
||||
- **Feeders run as `user: "0:0"`.** Their image bakes a non-root uid (10001) as
|
||||
owner of the state dir, but the platform injects `user: 1000:1000` on services
|
||||
without an explicit `user:`; running as root avoids a state-dir permission
|
||||
crash.
|
||||
- **A misconfigured feeder never crash-loops.** The feeder SDK keeps a
|
||||
feeder whose config is bad/incomplete *running* and reports it as **degraded**
|
||||
via `/health` (with the reason) rather than exiting — so one bad key can't take
|
||||
the gateway down. The app gates torznab on `service_started`, not
|
||||
`service_healthy`, for the same reason.
|
||||
@@ -0,0 +1,580 @@
|
||||
name: metagateway
|
||||
|
||||
# =============================================================================
|
||||
# MetaGateway — bridges external sources into the MetaMesh network.
|
||||
#
|
||||
# A single gateway peer — the bridging tier ONLY. Every content source is now a
|
||||
# SEPARATE store app (install whichever you want); the gateway discovers each by
|
||||
# URL over `pcs`, stores its resolved records, and soft-skips any not installed:
|
||||
# MetaFeeder · Gutenberg Project Gutenberg (+ Open Library cards)
|
||||
# MetaFeeder · arXiv / Europe PMC academic papers (Sci-Hub: separate opt-in app)
|
||||
# MetaFeeder · Wikimedia Commons open-licensed media (Giphy: separate opt-in app)
|
||||
# MetaFeeder · Prowlarr Prowlarr indexers (video/torrent) — own enrichment
|
||||
# MetaFeeder · Tribler Tribler IPv8 overlay (video/torrent) — own enrichment
|
||||
# Plus a dedicated meta-core, and a co-located meta-share INGESTER that announces the
|
||||
# gateway's resolved records on the swarm.
|
||||
#
|
||||
# Everything is scoped under the `metagateway-*` prefix and the per-stack
|
||||
# `metagateway` network + /DATA/AppData/metagateway/ data root, so this stack
|
||||
# installs alongside MetaWatch and the standalone Meta* apps with ZERO
|
||||
# container-name / data-path collisions. Its meta-core is private to the
|
||||
# gateway (holds resolved feeder records, not your library) — hence its own
|
||||
# core rather than the shared MetaCore app.
|
||||
#
|
||||
# Federation: the gateway-app joins the PUBLIC IPFS DHT and is discovered for
|
||||
# live fan-out on host port 4002; the meta-share ingester also joins the public
|
||||
# swarm (records + bitswap byte-serving) on host port 4005 + mDNS on `pcs`, so
|
||||
# MetaWatch peers find both the gateway and its seeded bytes.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ===========================================================================
|
||||
# metagateway-core — dedicated meta-core. Auto-store target for every resolved
|
||||
# feeder outcome; the gateway becomes a regular meta-core host of the CID.
|
||||
# File watcher ON so tmdb-plugin posters PUT over WebDAV get indexed + seeded.
|
||||
# ===========================================================================
|
||||
metagateway-core:
|
||||
# 1.0.1 carries the new meta-core API (records cids/ set, identity).
|
||||
image: ghcr.io/worph/meta-core:1.0.12
|
||||
container_name: metagateway-core
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 768m
|
||||
memswap_limit: 768m
|
||||
hostname: metagateway-core
|
||||
restart: unless-stopped
|
||||
privileged: true
|
||||
user: "0:0"
|
||||
cpu_shares: 50
|
||||
expose:
|
||||
- 9000
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/metacore:/meta-core:rw
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/files:/files:rw
|
||||
environment:
|
||||
- BASE_URL=http://metagateway-core:9000
|
||||
- TZ=$TZ
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://localhost:9000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
start_period: 30s
|
||||
# `pcs` so the external feeder apps can write records/posters to this core.
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
# ===========================================================================
|
||||
# metagateway-share — meta-share INGESTER role (META_CORE_URL set). Polls this
|
||||
# gateway's meta-core, announces resolved records on Kamilata + rebuilds its
|
||||
# Bloom filter, and seeds blobs over IPFS/bitswap so federated MetaWatch
|
||||
# queries reach the gateway's records. Public libp2p on host port 4005
|
||||
# (the gateway-app owns 4002 for discovery).
|
||||
# ===========================================================================
|
||||
metagateway-share:
|
||||
# 1.0.8 = rsa-enabled libp2p (joins the public IPFS DHT — see metagateway-app)
|
||||
# + P2P_ANNOUNCE support so remote peers can dial this bitswap seeder.
|
||||
image: ghcr.io/worph/meta-share:1.0.26
|
||||
container_name: metagateway-share
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 3g
|
||||
memswap_limit: 3g
|
||||
hostname: metagateway-share
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
cpu_shares: 30
|
||||
expose:
|
||||
- 3000
|
||||
# Own public libp2p port (4005). The gateway-app owns 4002 (discovery);
|
||||
# this ingester's bitswap tier serves the gateway's seeded bytes to remote
|
||||
# peers over the public DHT, so it needs its own dialable host port.
|
||||
ports:
|
||||
- "4005:4001"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/metacore:/meta-core:rw
|
||||
# Operator settings (settings.json — NNTP creds + indexer API keys). This
|
||||
# is the CREDENTIALED peer of the pair: it materialises NZB postings for
|
||||
# credential-less clients. Losing settings.json on an update means the
|
||||
# gateway silently stops serving nzb-release bytes until re-entered.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/share-config:/config:rw
|
||||
# Blockstore + seed manifest + NZB cache (META_SHARE_DATA=/data/meta-share).
|
||||
# Without it the gateway's seeded corpus is wiped on every recreate.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/share-data:/data:rw
|
||||
environment:
|
||||
P2P_LISTEN: "/ip4/0.0.0.0/tcp/4001"
|
||||
# Advertise the routable address (published host port 4005) so kad
|
||||
# provider records + identify carry a dialable multiaddr for remote
|
||||
# bitswap fetches. Behind Docker NAT the swarm otherwise only sees
|
||||
# container-internal addresses.
|
||||
# ${APP_PUBLIC_IPV4}, NOT ${APP_PUBLIC_IP} — see the note on the
|
||||
# gateway-app below: APP_PUBLIC_IP is the v6 address on an
|
||||
# IPv6-primary PCS, and `/ip4/<v6>/tcp/...` fails to parse.
|
||||
PUBLIC_ADDR: "/ip4/${APP_PUBLIC_IPV4}/tcp/4005"
|
||||
# Public IPFS DHT bootstraps by explicit /dns4/.../tcp/4001 (the nested
|
||||
# dnsaddr form resolves to wss-only in some container resolvers). rsa
|
||||
# feature required to complete the noise handshake with these RSA-keyed
|
||||
# (Qm...) bootstrap nodes.
|
||||
# Includes mars.i.ipfs.io — the only surviving legacy *.i.ipfs.io
|
||||
# bootstrapper (neptune/pluto/uranus/earth/saturn/venus are all NXDOMAIN).
|
||||
# It is here for resilience: the four IPFS-Foundation bootstrappers can
|
||||
# refuse an individual IP, and when they do a peer cannot join the DHT AT
|
||||
# ALL — no provider record, no discovery, no gateway fan-out. Observed on
|
||||
# watch.nsl.sh 2026-09-01: 2 connections with the four alone, 126 once a
|
||||
# reachable fifth was added. Costs nothing when the four work.
|
||||
KAD_BOOTSTRAP_PEERS: "/dns4/sg1.bootstrap.libp2p.io/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt,/dns4/sv15.bootstrap.libp2p.io/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN,/dns4/ny5.bootstrap.libp2p.io/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa,/dns4/am6.bootstrap.libp2p.io/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb,/dns4/mars.i.ipfs.io/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
|
||||
META_CORE_URL: "http://metagateway-core:9000"
|
||||
META_CORE_PATH: "/meta-core"
|
||||
BASE_URL: "http://metagateway-share:3000"
|
||||
# Renamed at the 1.0.26 bump (2026-08-26), as the old note here said to do:
|
||||
# META_SHARE_PEER_API_URL was the pre-1.0.8 spelling and now only logs a
|
||||
# deprecation warning. Do NOT set both.
|
||||
INTERNAL_BASE_URL: "http://metagateway-share:3000"
|
||||
TZ: "$TZ"
|
||||
depends_on:
|
||||
metagateway-core:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
# ===========================================================================
|
||||
# metagateway-app — the gateway peer (feeders-only). Discovers the feeder
|
||||
# sidecars from /data/meta-gateway/gateway-config.json (written by
|
||||
# pre-install-cmd), registers a RemoteFeederPlugin per upstream, and writes
|
||||
# resolved records to its meta-core. Dashboard fronted by the hash-lock below.
|
||||
# ===========================================================================
|
||||
metagateway-app:
|
||||
# 1.0.12 = rsa-enabled libp2p (required to join the public IPFS DHT: the
|
||||
# bootstrap.libp2p.io nodes are RSA-keyed, so without `rsa` the noise
|
||||
# handshake fails and the gateway never joins the DHT) + configurable
|
||||
# P2P listen/announce so it's discoverable + dialable off-host + the
|
||||
# 2026-07 discovery/filters fixes that make the meta-search fan-out
|
||||
# actually route `fileType:video` queries here.
|
||||
image: ghcr.io/worph/meta-gateway:1.0.25
|
||||
container_name: metagateway-app
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 1g
|
||||
memswap_limit: 1g
|
||||
hostname: metagateway-app
|
||||
restart: unless-stopped
|
||||
cpu_shares: 40
|
||||
expose:
|
||||
- 3000
|
||||
# Public libp2p listener (4002). This is the peer MetaWatch/meta-search
|
||||
# discovers via the kad `metamesh-gateway` provider namespace and dials
|
||||
# for live gateway fan-out (torznab/tribler). Must be host-published +
|
||||
# reachable inbound.
|
||||
ports:
|
||||
- "4002:4002"
|
||||
volumes:
|
||||
# gateway-config.json (feeders map) + gateway-enabled.json, written by
|
||||
# pre-install-cmd. Runtime redb caches also live here.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/state:/data/meta-gateway:rw
|
||||
environment:
|
||||
# NOTE: `meta_core_url` (and `meta_share_url`) live in gateway-config.json
|
||||
# since v1.0.12 — no longer read from env. The pre-install-cmd below seeds
|
||||
# them; the dashboard edits them at runtime. `META_GATEWAY_BASE_URL` stays
|
||||
# in env because it's an identify-layer setting (baseUrl= token in the
|
||||
# libp2p identify agent_version) — without it meta-search can't HTTP-fetch
|
||||
# /api/gateway/plugins on this gateway to populate its routing table.
|
||||
META_GATEWAY_PEER_ID: "metagateway"
|
||||
# MUST be resolvable BY A REMOTE PEER — it is advertised verbatim in the
|
||||
# identify `baseUrl=` token and used for nothing else in-process. A
|
||||
# docker-internal name here (the old `http://metagateway-app:3000`) makes
|
||||
# the gateway useless off-host even once kad discovery works: the remote
|
||||
# capability fetch fails, the snapshot stays empty, and
|
||||
# `upstreams_for_query_from_raw` then fans out to NOBODY. The
|
||||
# `/api/gateway/plugins` carve-out in the Caddy labels below is what makes
|
||||
# this URL answerable without a session.
|
||||
META_GATEWAY_BASE_URL: "https://metagateway-${APP_DOMAIN}"
|
||||
# Fixed libp2p port + advertise the routable address (published host
|
||||
# port 4002) so provider records + identify carry a dialable multiaddr.
|
||||
# Ceiling on established libp2p connections (new in meta-gateway 1.0.25).
|
||||
# Joining the public DHT is otherwise UNBOUNDED: at 1.0.24 this peer
|
||||
# climbed to ~600 connections and was OOM-killed by the memory limit
|
||||
# below roughly once a minute, and because each boot minted a fresh
|
||||
# peer-id, every provider record it published was stale within seconds —
|
||||
# remote peers only ever got "Unexpected peer ID". 128 is the default;
|
||||
# raise it only together with the memory limit.
|
||||
META_GATEWAY_MAX_CONNECTIONS: "128"
|
||||
META_GATEWAY_P2P_LISTEN: "/ip4/0.0.0.0/tcp/4002"
|
||||
# Setting a PUBLIC_ADDR is what flips the gateway out of local/mDNS-only
|
||||
# mode. Unset, meta-gateway pulls in NO kad bootstraps and sets
|
||||
# kad_provide_enabled=false, so it never publishes its `metamesh-gateway`
|
||||
# provider record and no off-host meta-search can discover it (it also
|
||||
# binds an ephemeral /tcp/0, leaving the `ports` mapping below with
|
||||
# nothing to forward to). See meta-gateway config.rs::resolve_network.
|
||||
# ${APP_PUBLIC_IPV4}, NOT ${APP_PUBLIC_IP}: on an IPv6-primary PCS,
|
||||
# APP_PUBLIC_IP holds the v6 address and `/ip4/<v6>/tcp/...` is not a
|
||||
# parseable multiaddr — the gateway refuses to start rather than
|
||||
# announce a lie. APP_PUBLIC_IPV4 is the v4 spelling, set on every box.
|
||||
META_GATEWAY_PUBLIC_ADDR: "/ip4/${APP_PUBLIC_IPV4}/tcp/4002"
|
||||
# Public IPFS DHT bootstraps via explicit /dns4/.../tcp/4001 (the nested
|
||||
# dnsaddr default resolves to wss-only in some container resolvers).
|
||||
# Includes mars.i.ipfs.io — the only surviving legacy *.i.ipfs.io
|
||||
# bootstrapper (neptune/pluto/uranus/earth/saturn/venus are all NXDOMAIN).
|
||||
# It is here for resilience: the four IPFS-Foundation bootstrappers can
|
||||
# refuse an individual IP, and when they do a peer cannot join the DHT AT
|
||||
# ALL — no provider record, no discovery, no gateway fan-out. Observed on
|
||||
# watch.nsl.sh 2026-09-01: 2 connections with the four alone, 126 once a
|
||||
# reachable fifth was added. Costs nothing when the four work.
|
||||
KAD_BOOTSTRAP_PEERS: "/dns4/sg1.bootstrap.libp2p.io/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt,/dns4/sv15.bootstrap.libp2p.io/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN,/dns4/ny5.bootstrap.libp2p.io/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa,/dns4/am6.bootstrap.libp2p.io/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb,/dns4/mars.i.ipfs.io/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
|
||||
TZ: "$TZ"
|
||||
depends_on:
|
||||
metagateway-core:
|
||||
condition: service_healthy
|
||||
# ALL feeders (book/paper/common + torznab/tribler) are SEPARATE store apps
|
||||
# now — the gateway discovers them by URL from its feeders config and
|
||||
# soft-skips any not installed, so they're not startup deps here.
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
# `pcs` so the gateway can reach the external feeder apps for discovery/query.
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
# NOTE: ALL feeders are now SEPARATE store apps, one per provider — e.g.
|
||||
# MetaFeeder · Gutenberg / arXiv / Europe PMC / Wikimedia Commons and the torrent
|
||||
# ones (MetaFeeder · Prowlarr / Tribler). The gateway is purely the bridging tier:
|
||||
# it discovers every feeder by URL (feeders config / dashboard) over `pcs`,
|
||||
# stores their resolved records in its meta-core, and announces them on the
|
||||
# swarm. Install whichever feeders you want; the gateway soft-skips any not
|
||||
# reachable. No feeder sidecars are bundled here anymore.
|
||||
|
||||
# ===========================================================================
|
||||
# OIDC perimeter for the MetaGateway dashboard. container_name MUST equal the
|
||||
# URL prefix (`metagateway`) for the registrar's redirect-URI allowlist.
|
||||
# ===========================================================================
|
||||
metagateway:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metagateway
|
||||
hostname: metagateway
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
BACKEND_HOST: "metagateway-app"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
depends_on:
|
||||
- metagateway-app
|
||||
labels:
|
||||
# ---- metagateway-${APP_DOMAIN} (gateway TLS) ----
|
||||
caddy_0: metagateway-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.0_handle: "/health"
|
||||
caddy_0.0_handle.reverse_proxy: "metagateway-app:3000"
|
||||
# Capability contract for REMOTE peers that have no session.
|
||||
# Exact-match path (no trailing `*`), so mutating routes like
|
||||
# `PUT /api/gateway/plugins/:id` stay behind AppShield.
|
||||
caddy_0.1_handle: "/api/gateway/plugins"
|
||||
caddy_0.1_handle.reverse_proxy: "metagateway-app:3000"
|
||||
caddy_0.2_handle: ""
|
||||
caddy_0.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metagateway-IP.nip.io (gateway TLS) ----
|
||||
caddy_1: metagateway-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.0_handle: "/health"
|
||||
caddy_1.0_handle.reverse_proxy: "metagateway-app:3000"
|
||||
# Capability contract for REMOTE peers that have no session.
|
||||
# Exact-match path (no trailing `*`), so mutating routes like
|
||||
# `PUT /api/gateway/plugins/:id` stay behind AppShield.
|
||||
caddy_1.1_handle: "/api/gateway/plugins"
|
||||
caddy_1.1_handle.reverse_proxy: "metagateway-app:3000"
|
||||
caddy_1.2_handle: ""
|
||||
caddy_1.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metagateway-IP.sslip.io (Let's Encrypt) ----
|
||||
caddy_2: metagateway-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.0_handle: "/health"
|
||||
caddy_2.0_handle.reverse_proxy: "metagateway-app:3000"
|
||||
# Capability contract for REMOTE peers that have no session.
|
||||
# Exact-match path (no trailing `*`), so mutating routes like
|
||||
# `PUT /api/gateway/plugins/:id` stay behind AppShield.
|
||||
caddy_2.1_handle: "/api/gateway/plugins"
|
||||
caddy_2.1_handle.reverse_proxy: "metagateway-app:3000"
|
||||
caddy_2.2_handle: ""
|
||||
caddy_2.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
# ===========================================================================
|
||||
# AppShield gates — expose the bundled service UIs behind Yundera OIDC SSO so
|
||||
# they're reachable + configurable from the dashboard's service links.
|
||||
# AppShield's container_name MUST equal its subdomain prefix (the auth-registrar
|
||||
# derives the OIDC client_id from it), so these use collision-free prefixes —
|
||||
# `metagateway-core`/`-share` are already the backend names. These backends
|
||||
# have no native auth, so they're reachable ONLY through these gates.
|
||||
# metagatewaycore → meta-core dashboard (nginx :80, NOT the :9000 API)
|
||||
# metagatewayshare → meta-share dashboard (:3000)
|
||||
# Prowlarr + Tribler are SEPARATE store apps (their own UIs + auth); the
|
||||
# feeders reach them over `pcs`, so there's no gate for them here.
|
||||
# ===========================================================================
|
||||
metagatewaycore:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metagatewaycore
|
||||
hostname: metagatewaycore
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose: ["80"]
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metagateway-core"
|
||||
BACKEND_PORT: "80"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metagatewaycore-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metagatewaycore-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metagatewaycore-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metagateway-core
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
metagatewayshare:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metagatewayshare
|
||||
hostname: metagatewayshare
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose: ["80"]
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metagateway-share"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metagatewayshare-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metagatewayshare-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metagatewayshare-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metagateway-share
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metagateway:
|
||||
name: metagateway
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
# `needs-public-ip` is the reserved functional tag: this app publishes a raw
|
||||
# libp2p TCP port that remote peers must dial inbound. Behind CGNAT or the
|
||||
# mesh-router tunnel it still installs and starts, but peers cannot reach it.
|
||||
tags: ["media", "needs-public-ip"]
|
||||
developer: Aptero Team
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**The source-bridging tier for the MetaMesh network.**
|
||||
|
||||
MetaGateway ingests content from external sources and makes it
|
||||
discoverable across the MetaMesh federation — so MetaWatch (and other
|
||||
MetaShare peers) can find and stream it. It's the **bridging tier only**:
|
||||
a dedicated meta-core that stores resolved records, plus a co-located
|
||||
meta-share ingester that announces them on the swarm. Every content source
|
||||
is a **separate MetaFeeder app**, one per provider (Prowlarr, Tribler, Usenet,
|
||||
Gutenberg, arXiv, Europe PMC, Wikimedia Commons, …)
|
||||
— install whichever you want; the gateway discovers each over the shared
|
||||
network and soft-skips any not installed.
|
||||
|
||||
**Sources (each a separate store app):**
|
||||
- **Prowlarr** - Torznab/Newznab indexers via the separate Prowlarr app (video/torrent/usenet)
|
||||
- **Tribler** - Decentralized Tribler discovery (video/torrent)
|
||||
- **Project Gutenberg** - Public-domain books (+ Open Library work cards)
|
||||
- **arXiv / Europe PMC** - Academic papers (Sci-Hub: separate opt-in app)
|
||||
- **Wikimedia Commons** - Open-licensed images (Giphy: separate opt-in app)
|
||||
|
||||
**How it fits:** MetaGateway is the *producer*; **MetaWatch** is the
|
||||
*consumer*. Install MetaGateway to populate the network, then browse it in
|
||||
MetaWatch. Self-contained — its meta-core is private to the gateway, so it
|
||||
coexists with MetaWatch and the standalone Meta* apps without conflicts.
|
||||
|
||||
**Enrichment:** Each resolved release is enriched with filename parsing and
|
||||
TMDB metadata (canonical content-addressed posters + plot) so it surfaces
|
||||
cleanly in MetaWatch's poster-gated wall.
|
||||
|
||||
fr_fr: |
|
||||
**La couche de passerelle de sources pour le réseau MetaMesh.**
|
||||
|
||||
MetaGateway ingère du contenu depuis des sources externes et le rend
|
||||
découvrable à travers la fédération MetaMesh — pour que MetaWatch (et les
|
||||
autres pairs MetaShare) puissent le trouver et le lire. Elle exécute tout
|
||||
le lot de feeders en sidecars, plus un Prowlarr intégré, un cœur Tribler
|
||||
headless, les plugins d'enrichissement meta-sort, un meta-core dédié et un
|
||||
ingesteur meta-share co-localisé qui annonce les enregistrements résolus.
|
||||
|
||||
**Sources (chacune une app du store) :**
|
||||
- **Prowlarr** - Indexeurs Torznab/Newznab via l'app Prowlarr (vidéo/torrent/usenet)
|
||||
- **Tribler** - Découverte décentralisée Tribler (vidéo/torrent)
|
||||
- **Project Gutenberg** - Livres du domaine public (+ fiches Open Library)
|
||||
- **arXiv / Europe PMC** - Articles académiques (Sci-Hub : app optionnelle séparée)
|
||||
- **Wikimedia Commons** - Images sous licence libre (Giphy : app optionnelle séparée)
|
||||
|
||||
**Place dans l'écosystème :** MetaGateway est le *producteur* ; **MetaWatch**
|
||||
est le *consommateur*. Installez MetaGateway pour alimenter le réseau, puis
|
||||
parcourez-le dans MetaWatch. Autonome — son meta-core est privé, donc il
|
||||
cohabite avec MetaWatch et les apps Meta* sans conflit.
|
||||
|
||||
tagline:
|
||||
en_us: Bridge torrent/usenet, books, papers and images into the MetaMesh network
|
||||
fr_fr: Connectez torrent/usenet, livres, articles et images au réseau MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaGateway
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/thumbnail.png
|
||||
|
||||
index: /
|
||||
store_app_id: metagateway
|
||||
main: metagateway
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Self-contained — no MetaCore required.** MetaGateway ships its own
|
||||
meta-core, ingester and feeders. Pair it with **MetaWatch** to browse
|
||||
what it surfaces.
|
||||
|
||||
**Authentication:** The gateway dashboard is protected by Yundera's
|
||||
Authelia via OIDC.
|
||||
|
||||
**The gateway works out of the box.** Books, papers and common-media
|
||||
feeders need no setup. Torrent/video needs the separate **Prowlarr** and
|
||||
**Tribler** apps — until they're installed + configured, those feeders
|
||||
simply show as **"degraded"** in the dashboard (they don't crash the app).
|
||||
|
||||
**Torznab / Prowlarr setup (required for torrent video):**
|
||||
1. Install the **Prowlarr** app from the store (it runs on the shared
|
||||
network as `prowlarr`).
|
||||
2. Open Prowlarr, complete its first-run wizard, then Indexers → Add
|
||||
Indexer → add one or more public indexers (confirm they test green).
|
||||
3. Settings → General → copy the **API Key**.
|
||||
4. In the gateway dashboard, open the **torznab** plugin → configure, paste
|
||||
the key into the indexer's API-key field, and Save. The torznab feeder
|
||||
auto-reloads and turns healthy.
|
||||
|
||||
**Tribler (optional, decentralized torrents):** install the **Tribler**
|
||||
app from the store; the gateway's tribler feeder connects to it
|
||||
automatically over the shared network — no key needed.
|
||||
|
||||
**TMDB enrichment:** set the TMDB v4 token in the torznab / tribler plugin
|
||||
config (dashboard) for canonical posters + overviews.
|
||||
|
||||
**Federation ports (optional).** The gateway-app host-publishes its
|
||||
libp2p listener on TCP **4002** (discovery / live fan-out) and the
|
||||
meta-share ingester on TCP **4005** (records + bitswap byte-serving) —
|
||||
open both inbound for best cross-node reach; same-host discovery (a
|
||||
MetaWatch on the same box) works over mDNS without them.
|
||||
|
||||
pre-install-cmd: |
|
||||
# Feeders map: all feeders are now SEPARATE store apps, listed by their pcs
|
||||
# URLs (MetaFeeder · Gutenberg / arXiv / Europe PMC / Wikimedia Commons /
|
||||
# Prowlarr / Tribler — the opt-in Sci-Hub and Giphy apps are not seeded). The
|
||||
# gateway soft-skips any feeder that isn't installed/reachable, and you can
|
||||
# add/remove feeders at runtime from the dashboard. Each key/URL MUST match
|
||||
# the corresponding app's feeder container hostname.
|
||||
# Guarded: this hook reruns on every reinstall and every version upgrade, and
|
||||
# the feeder map is editable from the dashboard — an unguarded `cat >` would
|
||||
# silently discard the operator's feeders each time.
|
||||
if [ ! -f /DATA/AppData/metagateway/state/gateway-config.json ]; then
|
||||
cat > /DATA/AppData/metagateway/state/gateway-config.json <<'JSON'
|
||||
{
|
||||
"meta_core_url": "http://metagateway-core:9000",
|
||||
"meta_share_url": "http://metagateway-share:3000",
|
||||
"feeders": {
|
||||
"gutenberg-feeder": { "url": "http://metafeedergutenberg-feeder:8080" },
|
||||
"arxiv-feeder": { "url": "http://metafeederarxiv-feeder:8080" },
|
||||
"europepmc-feeder": { "url": "http://metafeedereuropepmc-feeder:8080" },
|
||||
"wikicommons-feeder": { "url": "http://metafeederwikicommons-feeder:8080" },
|
||||
"prowlarr-feeder": { "url": "http://metafeederprowlarr-feeder:8080" },
|
||||
"tribler-feeder": { "url": "http://metafeedertribler-feeder:8080" }
|
||||
}
|
||||
}
|
||||
JSON
|
||||
fi
|
||||
if [ ! -f /DATA/AppData/metagateway/state/gateway-enabled.json ]; then
|
||||
cat > /DATA/AppData/metagateway/state/gateway-enabled.json <<'JSON'
|
||||
{
|
||||
"disabled": [],
|
||||
"enabled_opt_in": ["torznab", "tribler"]
|
||||
}
|
||||
JSON
|
||||
fi
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metagateway/metacore
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metagateway/files
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metagateway/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metagateway/share-config
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metagateway/share-data
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metagateway-${domain}
|
||||
webui-path: "/"
|
||||
|
After Width: | Height: | Size: 134 KiB |
@@ -0,0 +1,64 @@
|
||||
# MetaGateway — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metagateway-core` and `metagateway-share` run `user: 0:0`, and `metagateway-core` runs
|
||||
`privileged: true`.
|
||||
2. The stack publishes two host ports: `4002/tcp` (gateway discovery) and `4005/tcp`
|
||||
(meta-share ingester bitswap).
|
||||
3. The app exposes **three** public hostnames — `metagateway-`, `metagatewaycore-` and
|
||||
`metagatewayshare-` — each with its own AppShield sidecar, so two sidecar
|
||||
`container_name`s differ from the compose top-level `name:`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. `metagateway-core` is a bundled MetaCore: it runs rclone remote mounts, which need a
|
||||
mount namespace (`SYS_ADMIN` + `/dev/fuse` + an unconfined AppArmor profile), plus nginx
|
||||
and Redis. `metagateway-share` writes its service registration into the root-owned
|
||||
`/meta-core/services/`.
|
||||
2. libp2p is **not HTTP** — Caddy cannot proxy it, and remote peers resolve this node's
|
||||
public multiaddr from the kad-DHT and dial it directly. Without inbound reachability the
|
||||
node makes outbound connections only and never serves bytes, which is the app's function.
|
||||
The app therefore carries the reserved `needs-public-ip` tag.
|
||||
3. Each hostname is a distinct UI (gateway dashboard, core dashboard, share dashboard) and
|
||||
`auth-registrar` derives an app's OIDC `client_id` from the sidecar's container name via
|
||||
PTR lookup. Three routes need three stable, distinct names; only one of them can equal
|
||||
the top-level `name:`. `x-casaos.main` points at `metagateway`, the sidecar carrying the
|
||||
primary route.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Every publicly-routed hostname terminates at its own AppShield sidecar
|
||||
(`ghcr.io/yundera/appshield:2.0.9`) behind the PCS's Authelia SSO. Each sidecar sets
|
||||
`hostname:` equal to its `container_name` so `auth-registrar` can attest it via PTR and
|
||||
accept its redirect URIs; the names are app-prefixed, so they cannot collide with another
|
||||
app's services in the shared `pcs` DNS namespace.
|
||||
- The backends carry no Caddy labels and publish **no HTTP port** to the host. The only
|
||||
published ports are raw libp2p TCP.
|
||||
- libp2p transport is authenticated and encrypted (Noise) and peer identity is a public
|
||||
key, so an unauthenticated dial cannot read anything the node has not chosen to publish
|
||||
to the swarm.
|
||||
- No user directories are mounted — no `/DATA/Documents`, `/DATA/Downloads`, `/DATA/Media`
|
||||
or `/DATA/Gallery`.
|
||||
- `cpu_shares` on every service; memory-heavy services carry a hard `mem_limit` with
|
||||
`memswap_limit` equal to it. All image tags pinned.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Proxy libp2p through Caddy.** Not possible; the DHT advertises a raw TCP multiaddr.
|
||||
- **One sidecar with path-based routing to all three UIs.** The three backends generate
|
||||
absolute URLs and set cookies on their own hostnames; path-mounting them breaks both.
|
||||
- **Drop `privileged` from `metagateway-core` for a `cap_add` list.** The reduced form is
|
||||
most of what `privileged` grants and still failed to bring up the supervised daemons —
|
||||
see `Apps/MetaCore/rationale.md`, which is the same container.
|
||||
- **Split into separate store apps.** That is exactly what MetaCore / MetaShare /
|
||||
MetaGatewayShared are. This app is the self-contained variant, for a box that wants the
|
||||
gateway without the rest of the ecosystem installed.
|
||||
|
||||
## Data protection
|
||||
|
||||
All state lives under `/DATA/AppData/metagateway/`, declared in `x-compose-app.folders` and
|
||||
archived by Maison on uninstall. The two operator-editable config seeds
|
||||
(`gateway-config.json`, `gateway-enabled.json`) are written only when absent, so a reinstall
|
||||
or version upgrade preserves the feeder map the user configured from the dashboard. Only
|
||||
content published to the swarm leaves the box.
|
||||
|
After Width: | Height: | Size: 134 KiB |
@@ -0,0 +1,341 @@
|
||||
name: metagateway
|
||||
|
||||
# =============================================================================
|
||||
# MetaGateway (shared backend) — source-bridging tier for the MetaMesh network.
|
||||
#
|
||||
# This is the SHARED-BACKEND variant of MetaGateway. It ships only the gateway
|
||||
# dispatcher plus the card feeder, and REUSES the standalone MetaCore and
|
||||
# MetaShare apps already installed on the box:
|
||||
#
|
||||
# metagateway-app → the dispatcher / gateway UI
|
||||
# metagateway-card → the card feeder (curated-row cards for MetaWatch's wall)
|
||||
# metagateway → AppShield OIDC gate in front of the dispatcher
|
||||
#
|
||||
# Contrast with the `MetaGateway` app in this store, which is SELF-CONTAINED
|
||||
# (its own metagateway-core + metagateway-share + bundled Prowlarr/Tribler).
|
||||
# Pick exactly one: both use compose project `metagateway` and the same
|
||||
# /DATA/AppData/metagateway paths, so they cannot be installed side by side.
|
||||
#
|
||||
# Deployed and validated on watch.nsl.sh (2026-08-04).
|
||||
#
|
||||
# REQUIRES, on the same box and on the `pcs` network:
|
||||
# - MetaCore → provides metacore-app (record store the dispatcher writes to)
|
||||
# - MetaShare → provides metashare-app (seeds the CIDs the gateway resolves)
|
||||
# Indexer sources (torznab/Prowlarr and friends) are configured at runtime in
|
||||
# gateway-config.json under /DATA/AppData/metagateway/state — not here. Install
|
||||
# the separate **Prowlarr** and **MetaFeederProwlarr** apps to feed it.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metagateway-app — the gateway dispatcher. Resolves external-source queries
|
||||
# into MetaMesh records and writes them to the standalone metacore-app, whose
|
||||
# co-located metashare-app then seeds the resolved CIDs.
|
||||
# ---------------------------------------------------------------------------
|
||||
metagateway-app:
|
||||
image: ghcr.io/worph/meta-gateway:1.0.25
|
||||
container_name: metagateway-app
|
||||
hostname: metagateway-app
|
||||
restart: unless-stopped
|
||||
user: "1000:1000"
|
||||
cpu_shares: 40
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 1g
|
||||
memswap_limit: 1g
|
||||
expose:
|
||||
- 3000
|
||||
# Public libp2p listener. This is the peer a remote MetaWatch's meta-search
|
||||
# discovers via the kad `metamesh-gateway` provider namespace and dials for
|
||||
# live gateway fan-out. Must be host-published AND reachable inbound.
|
||||
ports:
|
||||
- "4002:4002"
|
||||
volumes:
|
||||
# gateway-config.json (source definitions, feeder URLs) + dispatcher state.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/state:/data/meta-gateway:rw
|
||||
environment:
|
||||
# Advertised verbatim in this peer's libp2p identify `agent_version`
|
||||
# (`baseUrl=...`) and used for NOTHING else in-process — see
|
||||
# meta-gateway `main.rs`: "prod compose would set it to the public
|
||||
# hostname behind whatever perimeter is in front."
|
||||
#
|
||||
# It MUST be resolvable BY A REMOTE PEER. It used to be
|
||||
# `http://metagateway-app:3000`, a docker-internal name, which made the
|
||||
# gateway useless off-host even once discovery worked: a remote
|
||||
# meta-search reads the served file_types/content_kinds by plain HTTP
|
||||
# `GET {baseUrl}/api/gateway/plugins` (the contract in the mirrored
|
||||
# `gateway_discovery.rs`), and an empty capability snapshot means
|
||||
# `upstreams_for_query_from_raw` fans out to NOBODY — every remote
|
||||
# typed query silently returns zero gateway hits. The `/api/gateway/plugins`
|
||||
# carve-out in the Caddy labels below is what makes this URL answerable.
|
||||
META_GATEWAY_BASE_URL: "https://metagateway-${APP_DOMAIN}"
|
||||
META_GATEWAY_PEER_ID: "metagateway"
|
||||
# --- cross-node discoverability (added 2026-08-26) ---------------------
|
||||
# WITHOUT META_GATEWAY_PUBLIC_ADDR meta-gateway starts in local/mDNS-only
|
||||
# mode: it loads NO kad bootstraps, sets kad_provide_enabled=false, and so
|
||||
# never publishes its `metamesh-gateway` provider record — no off-host
|
||||
# MetaWatch can EVER discover it, however open the firewall is. It also
|
||||
# binds an ephemeral /tcp/0, so the `ports` mapping above would have
|
||||
# nothing to forward to. It says so in its own first three log lines:
|
||||
# network_mode="local" kad_bootstrap_count=0 kad_provide_enabled=false
|
||||
# Verify after install: /api/peers → network_mode "public" + a non-empty
|
||||
# external_addrs. (This is the gap that made watch.nsl.sh invisible to the
|
||||
# wisera MetaWatch until 2026-08-26 — the self-contained MetaGateway
|
||||
# listing always had it; this shared-backend variant did not.)
|
||||
#
|
||||
# Use the IPV4 spelling, NOT the bare APP_PUBLIC_IP: on an IPv6-primary
|
||||
# PCS that variable holds the v6 address, and `/ip4/<v6>/tcp/...` is not a
|
||||
# parseable multiaddr — the gateway REFUSES TO START rather than announce
|
||||
# a lie. watch.nsl.sh is exactly such a box.
|
||||
# Ceiling on established libp2p connections (new in meta-gateway 1.0.25).
|
||||
# Joining the public DHT is otherwise UNBOUNDED: at 1.0.24 this peer
|
||||
# climbed to ~600 connections and was OOM-killed by the memory limit
|
||||
# below roughly once a minute, and because each boot minted a fresh
|
||||
# peer-id, every provider record it published was stale within seconds —
|
||||
# remote peers only ever got "Unexpected peer ID". 128 is the default;
|
||||
# raise it only together with the memory limit.
|
||||
META_GATEWAY_MAX_CONNECTIONS: "128"
|
||||
META_GATEWAY_P2P_LISTEN: "/ip4/0.0.0.0/tcp/4002"
|
||||
META_GATEWAY_PUBLIC_ADDR: "/ip4/${APP_PUBLIC_IPV4}/tcp/4002"
|
||||
# Public IPFS DHT bootstraps by explicit /dns4/.../tcp/4001 (the nested
|
||||
# dnsaddr default resolves to wss-only in some container resolvers).
|
||||
# Includes mars.i.ipfs.io — the only surviving legacy *.i.ipfs.io
|
||||
# bootstrapper (neptune/pluto/uranus/earth/saturn/venus are all NXDOMAIN).
|
||||
# It is here for resilience: the four IPFS-Foundation bootstrappers can
|
||||
# refuse an individual IP, and when they do a peer cannot join the DHT AT
|
||||
# ALL — no provider record, no discovery, no gateway fan-out. Observed on
|
||||
# watch.nsl.sh 2026-09-01: 2 connections with the four alone, 126 once a
|
||||
# reachable fifth was added. Costs nothing when the four work.
|
||||
KAD_BOOTSTRAP_PEERS: "/dns4/sg1.bootstrap.libp2p.io/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt,/dns4/sv15.bootstrap.libp2p.io/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN,/dns4/ny5.bootstrap.libp2p.io/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa,/dns4/am6.bootstrap.libp2p.io/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb,/dns4/mars.i.ipfs.io/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
|
||||
TZ: "$TZ"
|
||||
depends_on:
|
||||
metagateway-card:
|
||||
condition: service_healthy
|
||||
required: false
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
labels:
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/icon.png
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metagateway-card — the CARD feeder. Answers the curated-row card queries
|
||||
# that populate MetaWatch's home wall (one query per row, no per-title
|
||||
# anchoring). Optional: the dispatcher starts without it, but the wall on
|
||||
# every connected MetaWatch stays empty.
|
||||
#
|
||||
# Runs as 10001:999 — the uid/gid baked into the meta-feeder-tmdb image.
|
||||
# ---------------------------------------------------------------------------
|
||||
metagateway-card:
|
||||
image: ghcr.io/worph/meta-feeder-tmdb:1.0.2
|
||||
container_name: metagateway-card
|
||||
hostname: metagateway-card
|
||||
restart: unless-stopped
|
||||
user: "10001:999"
|
||||
cpu_shares: 30
|
||||
mem_limit: 256m
|
||||
memswap_limit: 256m
|
||||
expose:
|
||||
- 8080
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metagateway/card:/data/meta-feeder:rw
|
||||
environment:
|
||||
RUST_LOG: "info"
|
||||
TZ: "$TZ"
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:8080/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
labels:
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/icon.png
|
||||
networks:
|
||||
- metagateway
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# AppShield gate — the public entrypoint to the dispatcher UI, behind Yundera
|
||||
# OIDC SSO. container_name MUST equal the subdomain prefix (`metagateway`):
|
||||
# auth-registrar derives the OIDC client_id from it.
|
||||
#
|
||||
# /health is handled ahead of the gate so probes don't need a session.
|
||||
# `pcs` is REQUIRED — the gate resolves auth-registrar and casaos-oidc-bridge
|
||||
# by name on that network.
|
||||
# ---------------------------------------------------------------------------
|
||||
metagateway:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
container_name: metagateway
|
||||
hostname: metagateway
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
cpu_shares: 90
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
BACKEND_HOST: "metagateway-app"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
depends_on:
|
||||
- metagateway-app
|
||||
labels:
|
||||
# ---- metagateway-${APP_DOMAIN} (gateway TLS) ----
|
||||
caddy_0: metagateway-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.0_handle: "/health"
|
||||
caddy_0.0_handle.reverse_proxy: "metagateway-app:3000"
|
||||
# Gateway capability contract, read by REMOTE meta-search/meta-share
|
||||
# peers that have no session and never will. Exact-match path (no
|
||||
# trailing `*`), so the mutating `PUT /api/gateway/plugins/:id` and
|
||||
# every `/api/metadata/*` write stay behind AppShield.
|
||||
# Response is capability-only: plugin ids + file_types/content_kinds
|
||||
# + nzbFetch + nzbIndexers (indexer HOSTNAMES, no API keys).
|
||||
caddy_0.1_handle: "/api/gateway/plugins"
|
||||
caddy_0.1_handle.reverse_proxy: "metagateway-app:3000"
|
||||
caddy_0.2_handle: ""
|
||||
caddy_0.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metagateway-IP.nip.io (gateway TLS) ----
|
||||
caddy_1: metagateway-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.0_handle: "/health"
|
||||
caddy_1.0_handle.reverse_proxy: "metagateway-app:3000"
|
||||
# Gateway capability contract, read by REMOTE meta-search/meta-share
|
||||
# peers that have no session and never will. Exact-match path (no
|
||||
# trailing `*`), so the mutating `PUT /api/gateway/plugins/:id` and
|
||||
# every `/api/metadata/*` write stay behind AppShield.
|
||||
# Response is capability-only: plugin ids + file_types/content_kinds
|
||||
# + nzbFetch + nzbIndexers (indexer HOSTNAMES, no API keys).
|
||||
caddy_1.1_handle: "/api/gateway/plugins"
|
||||
caddy_1.1_handle.reverse_proxy: "metagateway-app:3000"
|
||||
caddy_1.2_handle: ""
|
||||
caddy_1.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metagateway-IP.sslip.io (Let's Encrypt) ----
|
||||
caddy_2: metagateway-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.0_handle: "/health"
|
||||
caddy_2.0_handle.reverse_proxy: "metagateway-app:3000"
|
||||
# Gateway capability contract, read by REMOTE meta-search/meta-share
|
||||
# peers that have no session and never will. Exact-match path (no
|
||||
# trailing `*`), so the mutating `PUT /api/gateway/plugins/:id` and
|
||||
# every `/api/metadata/*` write stay behind AppShield.
|
||||
# Response is capability-only: plugin ids + file_types/content_kinds
|
||||
# + nzbFetch + nzbIndexers (indexer HOSTNAMES, no API keys).
|
||||
caddy_2.1_handle: "/api/gateway/plugins"
|
||||
caddy_2.1_handle.reverse_proxy: "metagateway-app:3000"
|
||||
caddy_2.2_handle: ""
|
||||
caddy_2.2_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/icon.png
|
||||
networks:
|
||||
- metagateway
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metagateway:
|
||||
name: metagateway
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
hostname: metagateway-${APP_DOMAIN}
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaGateway/icon.png
|
||||
index: /
|
||||
is_uncontrolled: false
|
||||
main: metagateway
|
||||
port_map: "443"
|
||||
scheme: https
|
||||
store_app_id: metagatewayshared
|
||||
title:
|
||||
en_us: MetaGateway (shared backend)
|
||||
tagline:
|
||||
en_us: MetaGateway reusing the standalone MetaCore + MetaShare apps
|
||||
fr_fr: MetaGateway réutilisant les apps MetaCore et MetaShare autonomes
|
||||
description:
|
||||
en_us: |
|
||||
**The source-bridging tier for the MetaMesh network.**
|
||||
|
||||
MetaGateway ingests content from external sources and makes it
|
||||
discoverable across the MetaMesh federation — so MetaWatch (and other
|
||||
MetaShare peers) can find and stream it.
|
||||
|
||||
This build is the **shared-backend** variant: it ships the dispatcher and
|
||||
the card feeder only, and **requires** the standalone MetaCore and
|
||||
MetaShare apps on the same box, which it reuses for the record store and
|
||||
for seeding resolved CIDs. One core, one share — no duplicated services.
|
||||
If you want a stack that bundles its own core, share, Prowlarr and
|
||||
Tribler instead, install **MetaGateway**.
|
||||
|
||||
**How it fits:** MetaGateway is the *producer*; **MetaWatch** is the
|
||||
*consumer*. Install MetaGateway to populate the network, then browse it in
|
||||
MetaWatch.
|
||||
|
||||
**Sources** are configured at runtime in `gateway-config.json`; install
|
||||
**Prowlarr** and **MetaFeederProwlarr** (and optionally
|
||||
**MetaFeederUsenet**) to supply them.
|
||||
fr_fr: |
|
||||
**La couche de passerelle de sources pour le réseau MetaMesh.**
|
||||
|
||||
MetaGateway ingère du contenu depuis des sources externes et le rend
|
||||
découvrable à travers la fédération MetaMesh — pour que MetaWatch (et les
|
||||
autres pairs MetaShare) puissent le trouver et le lire.
|
||||
|
||||
Cette version est la variante **backend partagé** : elle n'embarque que le
|
||||
dispatcher et le feeder de cartes, et **nécessite** les apps MetaCore et
|
||||
MetaShare autonomes sur la même machine. Un seul cœur, un seul share —
|
||||
aucun doublon.
|
||||
|
||||
**Place dans l'écosystème :** MetaGateway est le *producteur* ; MetaWatch
|
||||
est le *consommateur*.
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Requires MetaCore + MetaShare.** Install both standalone apps first —
|
||||
this build reuses them instead of bundling its own copies.
|
||||
|
||||
**Conflicts with `MetaGateway`.** Both use the compose project
|
||||
`metagateway` and the same data paths. Install one or the other.
|
||||
|
||||
**Sources are runtime config.** After install, define your sources in
|
||||
`/DATA/AppData/metagateway/state/gateway-config.json` (or through the
|
||||
dispatcher UI). Install **Prowlarr** + **MetaFeederProwlarr** to give it
|
||||
something to query.
|
||||
|
||||
**The card feeder matters.** `metagateway-card` answers the curated-row
|
||||
card queries that fill MetaWatch's home wall. Without it the dispatcher
|
||||
still works for search, but every connected MetaWatch shows an empty
|
||||
wall.
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metagateway/state
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metagateway/card
|
||||
user: 10001
|
||||
group: 999
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metagateway-${domain}
|
||||
webui-path: "/"
|
||||
@@ -0,0 +1,52 @@
|
||||
# MetaGateway (shared backend) — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metagateway-card` runs as `10001:999` rather than `$PUID:$PGID`.
|
||||
2. The stack reuses the **standalone MetaCore and MetaShare apps'** services over the shared
|
||||
`pcs` network instead of bundling its own.
|
||||
3. The compose top-level `name:` is `metagateway`, the same project id as the
|
||||
self-contained `MetaGateway` app in this store.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. `10001:999` is the uid/gid the upstream meta-feeder-tmdb image runs its application as;
|
||||
its writable paths are baked to that owner. `/DATA/AppData/metagateway/card` is declared
|
||||
in `x-compose-app.folders` with `user: 10001` / `group: 999` so Maison creates it with
|
||||
the right ownership before the first start, instead of Compose creating it root-owned and
|
||||
the container failing on its first write.
|
||||
2. This is the point of the variant: on a box that already runs MetaCore and MetaShare,
|
||||
bundling a second Redis, leader-election tier and transport peer wastes memory and
|
||||
splits the record store in two.
|
||||
3. The two variants are alternatives, not companions — they share the project id and the
|
||||
same `/DATA/AppData/metagateway/` paths precisely so that installing one over the other
|
||||
reuses the same data rather than orphaning it. Exactly one may be installed at a time;
|
||||
this is stated in `tips.before_install`.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`)
|
||||
behind the PCS's Authelia SSO. The backends carry no Caddy labels and publish no host
|
||||
port.
|
||||
- No user directories are mounted — no `/DATA/Documents`, `/DATA/Downloads`, `/DATA/Media`
|
||||
or `/DATA/Gallery`.
|
||||
- All binds are under `/DATA/AppData/metagateway/`, declared in `x-compose-app.folders`.
|
||||
- `cpu_shares` on every service; image tags pinned; no `:latest`.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Run `metagateway-card` as `$PUID:$PGID`.** The upstream image's baked paths are owned by
|
||||
`10001:999`; it fails on first write.
|
||||
- **`chown` the directory in a hook instead.** The guidelines are explicit that a hook's
|
||||
`mkdir`/`chown` runs in the Maison container against a host path — the wrong filesystem.
|
||||
`folders` is the supported mechanism and it runs before the first pull.
|
||||
- **A distinct project id, so both variants can coexist.** They would then fight over the
|
||||
same public hostnames and the same record store; two gateways on one box is not a
|
||||
supported topology.
|
||||
|
||||
## Data protection
|
||||
|
||||
State is under `/DATA/AppData/metagateway/`, the unit Maison archives on uninstall and
|
||||
restores from backup. Because both MetaGateway variants use that path, switching between
|
||||
them preserves the user's configuration. The MetaCore and MetaShare data this variant reads
|
||||
belong to those apps and are archived with them.
|
||||
@@ -0,0 +1,43 @@
|
||||
# MetaRead
|
||||
|
||||
Reading client for the decentralized **MetaMesh** network — comics and manga,
|
||||
books, papers and scanned documents. The third client after MetaWatch (video)
|
||||
and MetaListen (audio), built the same way.
|
||||
|
||||
## What's in the stack
|
||||
|
||||
**Self-contained** (mirrors `packages/meta-read/dev/`). It does **not** depend
|
||||
on the standalone MetaCore app — it ships its own infra so it runs alongside
|
||||
MetaGateway and the other Meta* apps with no collisions:
|
||||
|
||||
| container | role |
|
||||
| --- | --- |
|
||||
| `metaread-core` | identity keystore, User Data Layer op-log, ingested records |
|
||||
| `metaread-search` | discovery — federated search, gateway fan-out, cover recovery |
|
||||
| `metaread-share` | pure content transport — byte Range, bitswap, BitTorrent |
|
||||
| `metaread-app` | the reader UI/back-end |
|
||||
| `metaread` | AppShield/OIDC gate (plus `metareadshare`, `metareadsearch` for the peer chips) |
|
||||
|
||||
⚠ **Host ports 4005 / 4006.** MetaWatch already publishes 4003/4004. Two peers
|
||||
cannot share a host port; walk the pairs upward for further clients.
|
||||
|
||||
## The reading model
|
||||
|
||||
The **depth of the hierarchy is a property of the item, not of the app**. A
|
||||
manga is *work → volume → chapter*, with several scanlations per chapter and a
|
||||
chooser between them. A Gutenberg novel is one work and one file — its page *is*
|
||||
the reader entry point, and no synthetic chapter 1 is invented to pad it.
|
||||
|
||||
Page decoding (CBZ, CBR, EPUB, PDF) happens **in the browser**. The server is a
|
||||
flat Range/206 byte proxy and never unpacks an archive on your behalf.
|
||||
|
||||
## It needs a feeder
|
||||
|
||||
MetaRead holds no catalogue. An empty library is the honest state of a mesh
|
||||
nobody has fed, not a fault. Install alongside **MetaGateway** plus:
|
||||
|
||||
- **MetaFeeder · Suwayomi** (+ **MetaFeeder · AniList**) — for comics and manga
|
||||
- **MetaFeeder · Gutenberg** — Project Gutenberg (+ Open Library cards), for public-domain books
|
||||
|
||||
and register each feeder in the gateway's config. Gateway feeder discovery is
|
||||
**startup-only**.
|
||||
@@ -0,0 +1,474 @@
|
||||
name: metaread
|
||||
|
||||
# =============================================================================
|
||||
# MetaRead — the READING client for the MetaMesh network.
|
||||
#
|
||||
# Comics and manga, books, papers and scanned documents. The third client on the
|
||||
# mesh after MetaWatch (video) and MetaListen (audio), and it is built the same
|
||||
# way: a self-contained stack with its OWN discovery (meta-search), transport
|
||||
# (meta-share) and meta-core, so it installs alongside MetaGateway and the
|
||||
# standalone Meta* apps with zero container-name / data-path collisions.
|
||||
#
|
||||
# metaread-core → /DATA/AppData/metaread/metacore (per-stack meta-core)
|
||||
# metaread-search → discovery (Kamilata federated search, identity proxy)
|
||||
# metaread-share → pure content transport (byte Range, bitswap, BT)
|
||||
# metaread-app → the reader UI/back-end (behind OIDC)
|
||||
#
|
||||
# ⚠ **Distinct libp2p host ports.** MetaWatch already publishes 4003/4004; this
|
||||
# stack uses 4005/4006. Two peers cannot share a host port, and the failure is
|
||||
# a container that will not start rather than anything subtle — but if you run
|
||||
# MetaListen too, keep walking the pairs upward.
|
||||
#
|
||||
# ── What it needs to actually show you something ─────────────────────────────
|
||||
# A client is a window onto the swarm; it holds no catalogue of its own. To see
|
||||
# comics you need a gateway peer running the literature feeders:
|
||||
#
|
||||
# **MetaFeeder · Suwayomi** — the chapters (and its own work cards), plus
|
||||
# optionally **MetaFeeder · AniList** to name the works. Install them beside
|
||||
# **MetaGateway** and register each feeder in the gateway's config.
|
||||
#
|
||||
# **MetaFeeder · Gutenberg** — Project Gutenberg (+ Open Library cards), for
|
||||
# public-domain books.
|
||||
#
|
||||
# Without either, MetaRead runs perfectly and shows an empty library — which is
|
||||
# the honest state of a mesh nobody has fed yet, not a fault.
|
||||
# =============================================================================
|
||||
|
||||
services:
|
||||
# ---------------------------------------------------------------------------
|
||||
# metaread-core — dedicated meta-core for this client: the identity keystore
|
||||
# (profiles are secp256k1 keypairs), the User Data Layer op-log, and whatever
|
||||
# records meta-search ingests off the swarm. File watcher OFF — MetaRead is a
|
||||
# consumer, not a library indexer.
|
||||
# ---------------------------------------------------------------------------
|
||||
metaread-core:
|
||||
image: ghcr.io/worph/meta-core:1.0.14
|
||||
container_name: metaread-core
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM instead of
|
||||
# swap.
|
||||
mem_limit: 768m
|
||||
memswap_limit: 768m
|
||||
hostname: metaread-core
|
||||
restart: unless-stopped
|
||||
privileged: true
|
||||
user: "0:0"
|
||||
cpu_shares: 30
|
||||
expose:
|
||||
- 9000
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/metacore:/meta-core:rw
|
||||
environment:
|
||||
- BASE_URL=http://metaread-core:9000
|
||||
- ENABLE_FILE_WATCHER=false
|
||||
- TZ=$TZ
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-q", "--spider", "http://localhost:9000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 30
|
||||
start_period: 30s
|
||||
networks:
|
||||
- metaread
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metaread-search — DISCOVERY. Federated search (Kamilata Bloom routing), the
|
||||
# gateway fan-out, cover recovery over bitswap, the shared prefs and the User
|
||||
# Data Layer. Public libp2p on host port 4006; mDNS on `pcs` for same-host
|
||||
# peers.
|
||||
#
|
||||
# ⚠ This is the peer that reaches MetaGateway. An empty MetaRead library with
|
||||
# a healthy gateway usually means this peer never joined the DHT — check its
|
||||
# peer count before suspecting the feeders.
|
||||
# ---------------------------------------------------------------------------
|
||||
metaread-search:
|
||||
image: ghcr.io/worph/meta-search:1.1.0
|
||||
container_name: metaread-search
|
||||
mem_limit: 1g
|
||||
memswap_limit: 1g
|
||||
hostname: metaread-search
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
cpu_shares: 30
|
||||
expose:
|
||||
- 3000
|
||||
ports:
|
||||
- "4006:4001"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/metacore:/meta-core:rw
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/search-data:/data:rw
|
||||
environment:
|
||||
P2P_LISTEN: "/ip4/0.0.0.0/tcp/4001"
|
||||
# ⚠ The IPV4 spelling, never the bare APP_PUBLIC_IP: on an IPv6-primary
|
||||
# PCS that variable holds the v6 address, and `/ip4/<v6>/tcp/...` is not a
|
||||
# parseable multiaddr — the peer REFUSES TO START rather than announce a
|
||||
# lie.
|
||||
PUBLIC_ADDR: "/ip4/${APP_PUBLIC_IPV4}/tcp/4006"
|
||||
# Explicit /dns4/ bootstraps: the nested dnsaddr default resolves to
|
||||
# wss-only in some container resolvers. mars.i.ipfs.io is the only
|
||||
# surviving legacy *.i.ipfs.io bootstrapper and is here for resilience —
|
||||
# the four IPFS-Foundation nodes can refuse an individual IP, and when they
|
||||
# do a peer cannot join the DHT AT ALL (no provider record, no discovery,
|
||||
# no gateway fan-out). Measured on watch.nsl.sh 2026-09-01: 2 connections
|
||||
# with the four alone, 126 once a reachable fifth was added.
|
||||
KAD_BOOTSTRAP_PEERS: "/dns4/sg1.bootstrap.libp2p.io/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt,/dns4/sv15.bootstrap.libp2p.io/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN,/dns4/ny5.bootstrap.libp2p.io/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa,/dns4/am6.bootstrap.libp2p.io/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb,/dns4/mars.i.ipfs.io/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
|
||||
META_CORE_URL: "http://metaread-core:9000"
|
||||
META_CORE_PATH: "/meta-core"
|
||||
BASE_URL: "http://metaread-search:3000"
|
||||
INTERNAL_BASE_URL: "http://metaread-search:3000"
|
||||
TZ: "$TZ"
|
||||
depends_on:
|
||||
metaread-core:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metaread
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metaread-share — PURE CONTENT TRANSPORT. Fetches and serves bytes by CID
|
||||
# over bitswap + a BitTorrent tier, and seeds whatever it fetches. Public
|
||||
# libp2p on host port 4005.
|
||||
#
|
||||
# ⚠ It must be reachable by whatever holds the pages. A comic chapter from the
|
||||
# aggregator tier is a list of per-page URL locators redeemed HERE, so a
|
||||
# Suwayomi behind auth or on a LAN this peer cannot reach yields a chapter
|
||||
# that lists correctly and will not open.
|
||||
# ---------------------------------------------------------------------------
|
||||
metaread-share:
|
||||
image: ghcr.io/worph/meta-share:1.0.27
|
||||
container_name: metaread-share
|
||||
# Smaller than MetaWatch's 3 GB on purpose: a chapter is tens of megabytes
|
||||
# where a film is gigabytes, and this peer never holds a whole video.
|
||||
mem_limit: 1500m
|
||||
memswap_limit: 1500m
|
||||
hostname: metaread-share
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
cpu_shares: 30
|
||||
expose:
|
||||
- 3000
|
||||
ports:
|
||||
- "4005:4001"
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/metacore:/meta-core:rw
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/metashare-config:/config:rw
|
||||
# The read cache: redb blockstore, seed manifest, sparse files. Without
|
||||
# this mount every recreate re-downloads from a cold swarm and the peer
|
||||
# seeds nothing back.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/share-data:/data:rw
|
||||
environment:
|
||||
P2P_LISTEN: "/ip4/0.0.0.0/tcp/4001"
|
||||
PUBLIC_ADDR: "/ip4/${APP_PUBLIC_IPV4}/tcp/4005"
|
||||
KAD_BOOTSTRAP_PEERS: "/dns4/sg1.bootstrap.libp2p.io/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt,/dns4/sv15.bootstrap.libp2p.io/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN,/dns4/ny5.bootstrap.libp2p.io/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa,/dns4/am6.bootstrap.libp2p.io/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb,/dns4/mars.i.ipfs.io/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ"
|
||||
META_CORE_URL: "http://metaread-core:9000"
|
||||
META_CORE_PATH: "/meta-core"
|
||||
BASE_URL: "http://metaread-share:3000"
|
||||
INTERNAL_BASE_URL: "http://metaread-share:3000"
|
||||
TZ: "$TZ"
|
||||
depends_on:
|
||||
metaread-core:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metaread
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# metaread-app — the reader. Zero page work server-side: a flat Range/206 byte
|
||||
# proxy to metaread-share, with the CBZ/EPUB/PDF decoding done in the browser.
|
||||
# My List, Continue Reading and reading positions live in the User Data Layer.
|
||||
# ---------------------------------------------------------------------------
|
||||
metaread-app:
|
||||
image: ghcr.io/worph/meta-read:1.0.0
|
||||
container_name: metaread-app
|
||||
hostname: metaread-app
|
||||
restart: unless-stopped
|
||||
cpu_shares: 30
|
||||
mem_limit: 512m
|
||||
memswap_limit: 512m
|
||||
expose:
|
||||
- 3000
|
||||
volumes:
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metaread/data:/app/data
|
||||
environment:
|
||||
META_SHARE_URL: "http://metaread-share:3000"
|
||||
META_SEARCH_URL: "http://metaread-search:3000"
|
||||
# Browser-reachable topbar chips — the AppShield gates below expose the
|
||||
# co-located transport and discovery peers behind Yundera OIDC.
|
||||
META_SHARE_PUBLIC_URL: "https://metareadshare-${APP_DOMAIN}"
|
||||
META_SEARCH_PUBLIC_URL: "https://metareadsearch-${APP_DOMAIN}"
|
||||
TZ: "$TZ"
|
||||
depends_on:
|
||||
metaread-search:
|
||||
condition: service_healthy
|
||||
metaread-share:
|
||||
condition: service_healthy
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 5s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 5s
|
||||
networks:
|
||||
- metaread
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# OIDC perimeter for the reader UI. container_name MUST equal the URL prefix
|
||||
# so the registrar's redirect-URI allowlist matches `metaread-<user>.<domain>`.
|
||||
# ---------------------------------------------------------------------------
|
||||
metaread:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metaread
|
||||
hostname: metaread
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metaread-app"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metaread-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metaread-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metaread-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metaread-app
|
||||
networks:
|
||||
- metaread
|
||||
- pcs
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Gates for the two co-located peers, so the reader's topbar chips resolve to
|
||||
# something a browser can open. Diagnostics only — nothing routes through them.
|
||||
# ---------------------------------------------------------------------------
|
||||
metareadshare:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 50
|
||||
container_name: metareadshare
|
||||
hostname: metareadshare
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metaread-share"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metareadshare-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metareadshare-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metareadshare-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metaread-share
|
||||
networks:
|
||||
- metaread
|
||||
- pcs
|
||||
|
||||
metareadsearch:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 50
|
||||
container_name: metareadsearch
|
||||
hostname: metareadsearch
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
expose:
|
||||
- 80
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
BACKEND_HOST: "metaread-search"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
labels:
|
||||
caddy_0: metareadsearch-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_1: metareadsearch-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.reverse_proxy: "{{upstreams 80}}"
|
||||
caddy_2: metareadsearch-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.reverse_proxy: "{{upstreams 80}}"
|
||||
depends_on:
|
||||
- metaread-search
|
||||
networks:
|
||||
- metaread
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
metaread:
|
||||
name: metaread
|
||||
driver: bridge
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
main: metaread
|
||||
webui_port: 80
|
||||
index: /
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**A reading client for the MetaMesh network — comics and manga, books,
|
||||
papers and scanned documents.**
|
||||
|
||||
MetaRead is to reading what **MetaWatch** is to video: a library you browse
|
||||
by *work* rather than by filename. Search returns a grid of covers, a work
|
||||
opens to its chapter or edition list, and a chapter opens in a reader that
|
||||
does the page decoding **in your browser** — the server is a flat byte
|
||||
proxy and never unpacks an archive on your behalf.
|
||||
|
||||
**What it reads.** CBZ and CBR comics, EPUB and PDF books, and the
|
||||
per-page image lists that a manga aggregator serves for a chapter nobody
|
||||
has downloaded yet. Reading direction, page layout and spread are per-work
|
||||
and remembered.
|
||||
|
||||
**How the catalogue works.** The depth of the hierarchy is a property of
|
||||
the item, not of the app. A manga is *work → volume → chapter*, with
|
||||
several scanlations per chapter and a chooser to pick between them. A
|
||||
Gutenberg novel is one work and one file — its page *is* the reader entry
|
||||
point, with no synthetic chapter 1 invented to pad it out.
|
||||
|
||||
**Languages.** Pick the languages you read once, in the top bar, and every
|
||||
view honours it — the same preference MetaWatch and the meta-share
|
||||
dashboard already speak. An item that declares no language is always
|
||||
shown: absence means *unknown*, never *not yours*.
|
||||
|
||||
**Self-contained.** Ships its own discovery, transport and meta-core, so it
|
||||
installs beside **MetaGateway** and the other Meta* apps with no
|
||||
collisions. Your profile is a keypair, and your list, positions and
|
||||
ratings live in the User Data Layer rather than on someone's server.
|
||||
|
||||
**Requires a feeder to have anything to show:** **MetaFeeder · Suwayomi**
|
||||
(with **MetaFeeder · AniList**) for comics and manga, **MetaFeeder · Gutenberg**
|
||||
for public-domain books. Without one it runs perfectly and shows an empty
|
||||
library — the honest state of a mesh nobody has fed.
|
||||
|
||||
fr_fr: |
|
||||
**Un client de lecture pour le réseau MetaMesh — BD et manga, livres,
|
||||
articles et documents numérisés.**
|
||||
|
||||
MetaRead est à la lecture ce que **MetaWatch** est à la vidéo : une
|
||||
bibliothèque que l'on parcourt par *œuvre* plutôt que par nom de fichier.
|
||||
La recherche renvoie une grille de couvertures, une œuvre s'ouvre sur sa
|
||||
liste de chapitres ou d'éditions, et un chapitre s'ouvre dans un lecteur
|
||||
qui décode les pages **dans votre navigateur** — le serveur n'est qu'un
|
||||
relais d'octets et n'ouvre jamais une archive à votre place.
|
||||
|
||||
**Ce qu'il lit.** BD en CBZ et CBR, livres en EPUB et PDF, et les listes
|
||||
d'images par page qu'un agrégateur de manga sert pour un chapitre que
|
||||
personne n'a encore téléchargé. Sens de lecture, mise en page et double
|
||||
page sont propres à chaque œuvre et mémorisés.
|
||||
|
||||
**La profondeur du catalogue appartient à l'œuvre, pas à l'app.** Un manga
|
||||
est *œuvre → volume → chapitre*, avec plusieurs scanlations par chapitre et
|
||||
un sélecteur pour choisir. Un roman de Gutenberg est une œuvre et un
|
||||
fichier — sa page *est* le point d'entrée du lecteur, sans chapitre 1
|
||||
artificiel inventé pour meubler.
|
||||
|
||||
**Langues.** Choisissez une fois les langues que vous lisez, dans la barre
|
||||
du haut, et toutes les vues la respectent. Un élément sans langue déclarée
|
||||
reste affiché : l'absence signifie *inconnu*, jamais *pas pour vous*.
|
||||
|
||||
**Autonome.** Embarque sa propre découverte, son transport et son
|
||||
meta-core. Votre profil est une paire de clés ; listes, positions et notes
|
||||
vivent dans la User Data Layer.
|
||||
|
||||
**Nécessite un feeder** pour avoir quelque chose à afficher :
|
||||
**MetaFeeder · Suwayomi** (avec **MetaFeeder · AniList**) pour la BD et le manga,
|
||||
**MetaFeeder · Gutenberg** pour les livres du domaine public.
|
||||
|
||||
tagline:
|
||||
en_us: Read comics, manga and books from the MetaMesh network
|
||||
fr_fr: Lisez BD, manga et livres depuis le réseau MetaMesh
|
||||
|
||||
title:
|
||||
en_us: MetaRead
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaRead/icon.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaRead/thumbnail.png
|
||||
|
||||
store_app_id: metaread
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Ports.** This stack publishes libp2p on host ports **4005** and
|
||||
**4006**. MetaWatch already uses 4003/4004 — if you run another Meta*
|
||||
client, keep walking the pairs upward. Two peers cannot share a host
|
||||
port.
|
||||
|
||||
**Install order:**
|
||||
1. **MetaGateway** — the peer that runs the feeders and announces what
|
||||
they find.
|
||||
2. **MetaFeeder · Suwayomi** (+ **MetaFeeder · AniList**) and/or
|
||||
**MetaFeeder · Gutenberg**, then register
|
||||
each in the gateway's *Add a feeder* box. Gateway feeder discovery is
|
||||
**startup-only**, so the gateway restarts itself afterwards.
|
||||
3. This app.
|
||||
|
||||
**First run.** Open MetaRead and create a profile — it is a keypair
|
||||
generated in your browser, and the secret key is shown once. Save it: it
|
||||
is the only way back in, and nobody can reset it for you.
|
||||
|
||||
**An empty library is not a fault.** MetaRead holds no catalogue of its
|
||||
own. If the grid is empty, the question is whether a feeder is attached
|
||||
and whether this stack's discovery peer has joined the DHT — the
|
||||
*meta-search* chip in the top bar shows its peer count.
|
||||
|
||||
**Authentication:** the UI is behind Yundera's Authelia (OIDC), and your
|
||||
reading profile is a second, separate identity inside it.
|
||||
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
folders:
|
||||
- path: /DATA/AppData/metaread/metacore
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metaread/data
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metaread/share-data
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metaread/search-data
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metaread/metashare-config
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
webui-host: metaread-${domain}
|
||||
webui-path: "/"
|
||||
|
After Width: | Height: | Size: 20 KiB |
|
After Width: | Height: | Size: 20 KiB |
@@ -0,0 +1,341 @@
|
||||
name: metashare
|
||||
|
||||
services:
|
||||
# Backend service. Reachable on the pcs network as `metashare-app`.
|
||||
# Public ingress goes through the `metashare` hash-lock proxy below.
|
||||
#
|
||||
# Unlike the other meta apps, this container is a self-contained Rust
|
||||
# binary serving Axum on :3000 directly (no nginx wrapper, no supervisord).
|
||||
# Ports 3000 and 4001 are split:
|
||||
# - 3000 HTTP API + dashboard (proxied through the hash-lock + Caddy)
|
||||
# - 4001 libp2p TCP listener — must be reachable on the public IP so
|
||||
# other archivists' peers can dial in via kad-DHT provider records.
|
||||
metashare-app:
|
||||
# 1.0.13 = nocopy library seeding (meta-core files seeded as filestore refs,
|
||||
# no second on-disk copy) + core/cache source surfacing in the dashboard, on
|
||||
# top of the 1.0.8 floor this pin still requires: rsa-enabled libp2p
|
||||
# (required to join the public IPFS DHT — the bootstrap.libp2p.io nodes are
|
||||
# RSA-keyed; without `rsa` the noise handshake fails and this peer never
|
||||
# actually reaches the DHT) + PUBLIC_ADDR support so remote archivists can
|
||||
# dial this seeder.
|
||||
image: ghcr.io/worph/meta-share:1.0.26
|
||||
container_name: metashare-app
|
||||
# Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
|
||||
# instead of swap. Mirrors the dev sub-stack limits.
|
||||
mem_limit: 3g
|
||||
memswap_limit: 3g
|
||||
hostname: metashare-app
|
||||
restart: unless-stopped
|
||||
cpu_shares: 20
|
||||
|
||||
# Give meta-share room to shut down cleanly on SIGTERM (docker stop /
|
||||
# `up -d` recreate) before docker escalates to SIGKILL. The binary catches
|
||||
# SIGTERM, drains in-flight HTTP briefly, then unwinds so redb's Database
|
||||
# Drop writes its allocator-state table — without which the next open runs a
|
||||
# full multi-scan repair of blocks.redb (minutes of silent CPU before the
|
||||
# port binds, proportional to blockstore size). 3s drain + the allocator
|
||||
# write fits the 10s default, but a large store gets headroom here.
|
||||
stop_grace_period: 30s
|
||||
|
||||
expose:
|
||||
- 3000 # HTTP API (Axum)
|
||||
|
||||
# libp2p TCP must be host-published. Caddy can't proxy it (not HTTP)
|
||||
# and other peers' kad-DHT lookups dial the public multiaddr directly.
|
||||
ports:
|
||||
- "4001:4001"
|
||||
|
||||
volumes:
|
||||
# Shared infrastructure state (leader election, Redis dump, service discovery)
|
||||
# Owned by MetaCore; mounted rw here so meta-share registers under services/.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metacore/meta-core:/meta-core:rw
|
||||
# Operator settings (META_SHARE_CONFIG_DIR, default /config): settings.json
|
||||
# holds the NNTP credentials + per-indexer API keys entered in the dashboard.
|
||||
# MUST be persisted — without this mount they live on the container's
|
||||
# writable layer and are lost on every image bump / recreate, forcing the
|
||||
# operator to re-enter their credentials after each update.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metashare/config:/config:rw
|
||||
# Seeded bytes + the seed index (META_SHARE_DATA, default /data/meta-share):
|
||||
# the redb IPFS blockstore, seed manifest, librqbit sparse torrent files
|
||||
# (bt-temp/), the NZB cache and the ipfs-seed cursor. Recomputable, but a
|
||||
# peer whose job is "fetch implies seed" comes back seeding NOTHING after a
|
||||
# recreate without this, and re-downloads everything from a cold swarm.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metashare/data:/data:rw
|
||||
|
||||
# Note: no /files mounts — meta-share resolves file paths through MetaCore's
|
||||
# WebDAV URL (Architecture V3). META_SHARE_FILES_PATH is only used as the
|
||||
# prefix for translating absolute filePaths back to WebDAV-relative paths.
|
||||
|
||||
environment:
|
||||
- P2P_LISTEN=/ip4/0.0.0.0/tcp/4001
|
||||
# Advertise the routable address (published host port 4001) so kad
|
||||
# provider records + identify carry a dialable multiaddr — otherwise,
|
||||
# behind Docker NAT, the swarm only sees container-internal addresses
|
||||
# and remote archivists can't dial in to fetch seeded blocks.
|
||||
# ${APP_PUBLIC_IPV4}, NOT ${APP_PUBLIC_IP}: on an IPv6-primary PCS
|
||||
# APP_PUBLIC_IP holds the v6 address, and `/ip4/<v6>/tcp/...` is not a
|
||||
# parseable multiaddr — meta-share REFUSES TO START rather than announce
|
||||
# a lie (config.rs: "PUBLIC_ADDR is not a valid multiaddr").
|
||||
- PUBLIC_ADDR=/ip4/${APP_PUBLIC_IPV4}/tcp/4001
|
||||
# Public IPFS DHT bootstraps by explicit /dns4/.../tcp/4001. The default
|
||||
# nested-dnsaddr form resolves to wss-only in some container resolvers
|
||||
# (no ws transport → no bootstrap); these direct addresses avoid that.
|
||||
# Includes mars.i.ipfs.io — the only surviving legacy *.i.ipfs.io
|
||||
# bootstrapper. Resilience: the four IPFS-Foundation bootstrappers can refuse
|
||||
# an individual IP, and when they do a peer cannot join the DHT AT ALL.
|
||||
# Observed on watch.nsl.sh 2026-09-01: 2 connections with the four alone,
|
||||
# 126 once a reachable fifth was added.
|
||||
- KAD_BOOTSTRAP_PEERS=/dns4/sg1.bootstrap.libp2p.io/tcp/4001/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt,/dns4/sv15.bootstrap.libp2p.io/tcp/4001/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN,/dns4/ny5.bootstrap.libp2p.io/tcp/4001/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa,/dns4/am6.bootstrap.libp2p.io/tcp/4001/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb,/dns4/mars.i.ipfs.io/tcp/4001/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ
|
||||
|
||||
# Service registration into /meta-core/services/
|
||||
- META_CORE_PATH=/meta-core
|
||||
- BASE_URL=https://metashare-$APP_DOMAIN
|
||||
|
||||
# meta-core ingestion — the local archivist's library. This peer pulls
|
||||
# records from MetaCore and announces them on the Kamilata swarm.
|
||||
- META_CORE_URL=http://metacore-app:80
|
||||
|
||||
# Inter-peer fetch URL — where another archivist's meta-share reaches
|
||||
# this peer's HTTP API. Same as BASE_URL for prod (other peers fetch
|
||||
# via the public Caddy URL, which routes back through this hash-lock
|
||||
# proxy with /api/peer/* and /api/file/* on the bypass list).
|
||||
#
|
||||
# Renamed at the 1.0.24 bump (2026-08-04), as the old note here said to do:
|
||||
# META_SHARE_PEER_API_URL was the pre-1.0.8 spelling. watch.nsl.sh runs
|
||||
# 1.0.24 with INTERNAL_BASE_URL and registers its peer_url correctly, so the
|
||||
# deprecated name is no longer needed. Do NOT set both.
|
||||
- INTERNAL_BASE_URL=https://metashare-$APP_DOMAIN
|
||||
|
||||
# NOTE: KAD_NAMESPACE, TARGET_PEER_COUNT, ENABLE_MDNS, META_CORE_POLL_SECS,
|
||||
# META_CORE_MAX_RECORDS, HTTP_LISTEN, SERVICE_NAME and RUST_LOG were all
|
||||
# dropped — each merely restated the code default in meta-share:1.0.8.
|
||||
# Set KAD_NAMESPACE to a non-default value to isolate a private cohort
|
||||
# (every peer in the cohort must agree on it).
|
||||
|
||||
- TZ=$TZ
|
||||
|
||||
# Runs as root because the meta-core volume is provisioned mode 0755 by
|
||||
# MetaCore (root). Same convention as the other meta apps.
|
||||
user: 0:0
|
||||
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-sf", "http://localhost:3000/health"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 10
|
||||
start_period: 30s
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
# OIDC perimeter for the MetaShare dashboard. Self-registers as an OIDC client
|
||||
# with the platform's auth-registrar on first login (no secrets to manage).
|
||||
#
|
||||
# Naming: container_name MUST equal the URL-prefix of the public hostname so
|
||||
# the registrar's redirect-URI allowlist accepts `metashare-<user>.<domain>`.
|
||||
metashare:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metashare
|
||||
hostname: metashare
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
BACKEND_HOST: "metashare-app"
|
||||
BACKEND_PORT: "3000"
|
||||
LISTEN_PORT: "80"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
|
||||
depends_on:
|
||||
- metashare-app
|
||||
|
||||
labels:
|
||||
# Peer-to-peer surface bypasses the hash-lock so other archivists'
|
||||
# meta-share peers can reach /api/peers, /api/peer/*, /api/file/*,
|
||||
# /api/search* and /api/services without an OIDC session. The
|
||||
# dashboard (everything else) sits behind hash-lock as usual.
|
||||
|
||||
# ---- metashare-${APP_DOMAIN} (gateway TLS) ----
|
||||
caddy_0: metashare-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.0_handle: "/health"
|
||||
caddy_0.0_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_0.1_handle: "/api/peers*"
|
||||
caddy_0.1_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_0.2_handle: "/api/peer/*"
|
||||
caddy_0.2_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_0.3_handle: "/api/file/*"
|
||||
caddy_0.3_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_0.4_handle: "/api/search*"
|
||||
caddy_0.4_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_0.5_handle: "/api/services"
|
||||
caddy_0.5_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_0.6_handle: ""
|
||||
caddy_0.6_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metashare-IP.nip.io (gateway TLS) ----
|
||||
caddy_1: metashare-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.0_handle: "/health"
|
||||
caddy_1.0_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_1.1_handle: "/api/peers*"
|
||||
caddy_1.1_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_1.2_handle: "/api/peer/*"
|
||||
caddy_1.2_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_1.3_handle: "/api/file/*"
|
||||
caddy_1.3_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_1.4_handle: "/api/search*"
|
||||
caddy_1.4_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_1.5_handle: "/api/services"
|
||||
caddy_1.5_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_1.6_handle: ""
|
||||
caddy_1.6_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metashare-IP.sslip.io (Let's Encrypt) ----
|
||||
caddy_2: metashare-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.0_handle: "/health"
|
||||
caddy_2.0_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_2.1_handle: "/api/peers*"
|
||||
caddy_2.1_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_2.2_handle: "/api/peer/*"
|
||||
caddy_2.2_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_2.3_handle: "/api/file/*"
|
||||
caddy_2.3_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_2.4_handle: "/api/search*"
|
||||
caddy_2.4_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_2.5_handle: "/api/services"
|
||||
caddy_2.5_handle.reverse_proxy: "metashare-app:3000"
|
||||
caddy_2.6_handle: ""
|
||||
caddy_2.6_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
# `needs-public-ip` is the reserved functional tag: this app publishes a raw
|
||||
# libp2p TCP port that remote peers must dial inbound. Behind CGNAT or the
|
||||
# mesh-router tunnel it still installs and starts, but peers cannot reach it.
|
||||
tags: ["media", "needs-public-ip"]
|
||||
developer: Aptero Team
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Decentralized metadata sharing across MetaMesh archivists.**
|
||||
|
||||
MetaShare turns your local MetaMesh library into a peer in a federated discovery network. Archivists running their own Yundera nodes can search across each other's libraries without surrendering control of their files — discovery returns pointers (peer + CID), and any retrieval is an explicit HTTP fetch from the source peer.
|
||||
|
||||
**Key Features:**
|
||||
- **Federated Search** - Type-once, query-many across the cohort using the Kamilata routing primitive
|
||||
- **Pointer-Only Discovery** - Search returns `(peer, cid)`; data never leaves the source until you ask for it
|
||||
- **Automatic Peer Discovery** - Public IPFS DHT (kad) finds other archivists; mDNS handles same-LAN cohorts; no designated bootstrap node
|
||||
- **Cohort Scoping** - `KAD_NAMESPACE` lets a private group of archivists isolate their swarm from the public default
|
||||
- **Built on libp2p** - QUIC + TCP transport, Noise-encrypted, peer-id authenticated
|
||||
- **MetaCore-Backed** - Ingests from your local MetaCore Redis and re-publishes records on the swarm
|
||||
|
||||
**Requires:** MetaCore (install first; MetaShare reads its metadata from MetaCore's API)
|
||||
|
||||
**Works with:** MetaSort (so MetaCore has records to share), MetaFuse, MetaStremio
|
||||
|
||||
**Status:** v0.2 POC. In-memory record store (rebuilt from MetaCore on each start), no persistent peer-id, no announcement signing yet. Production-grade hardening lands in v0.2.1.
|
||||
|
||||
**Dashboard Features:** Peer list with live connection status, federated search, per-record source attribution, swarm health diagnostics
|
||||
|
||||
fr_fr: |
|
||||
**Partage de metadonnees decentralise entre archivistes MetaMesh.**
|
||||
|
||||
MetaShare transforme votre bibliotheque MetaMesh locale en pair d'un reseau de decouverte federe. Les archivistes faisant tourner leur propre serveur Yundera peuvent chercher dans les bibliotheques des autres sans abandonner le controle de leurs fichiers — la decouverte renvoie des pointeurs (pair + CID), et toute recuperation est une requete HTTP explicite vers le pair source.
|
||||
|
||||
**Fonctionnalites Cles:**
|
||||
- **Recherche Federee** - Une seule requete interroge tout le cohorte via la primitive de routage Kamilata
|
||||
- **Decouverte par Pointeurs** - La recherche renvoie `(pair, cid)` ; les donnees ne quittent le pair source que sur demande explicite
|
||||
- **Decouverte Automatique des Pairs** - Le DHT IPFS public (kad) trouve les autres archivistes ; mDNS gere les cohortes sur le meme LAN ; aucun noeud bootstrap designe
|
||||
- **Isolation par Cohorte** - `KAD_NAMESPACE` permet a un groupe prive d'archivistes d'isoler leur swarm du namespace public
|
||||
- **Construit sur libp2p** - Transport QUIC + TCP, chiffrement Noise, authentification par peer-id
|
||||
- **Adosse a MetaCore** - Ingere depuis votre Redis MetaCore local et republie les enregistrements sur le swarm
|
||||
|
||||
**Necessite:** MetaCore (installer en premier ; MetaShare lit ses metadonnees depuis l'API MetaCore)
|
||||
|
||||
**Fonctionne avec:** MetaSort (pour que MetaCore ait des enregistrements a partager), MetaFuse, MetaStremio
|
||||
|
||||
**Statut:** POC v0.2. Stockage des enregistrements en memoire (reconstruit depuis MetaCore a chaque demarrage), pas de peer-id persistant, pas encore de signature des annonces. Le durcissement de niveau production arrive en v0.2.1.
|
||||
|
||||
tagline:
|
||||
en_us: Decentralized metadata sharing - Federated search across archivists without surrendering your files
|
||||
fr_fr: Partage de metadonnees decentralise - Recherche federee entre archivistes sans abandonner vos fichiers
|
||||
|
||||
title:
|
||||
en_us: MetaShare
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaShare/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaShare/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaShare/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaShare/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaShare/thumbnail.png
|
||||
|
||||
index: /
|
||||
store_app_id: metashare
|
||||
main: metashare
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Requires MetaCore** - Install MetaCore first.
|
||||
|
||||
**Port 4001 must be reachable from the public internet.** MetaShare's libp2p
|
||||
TCP listener is host-published on port 4001 so other archivists' peers
|
||||
(and the public IPFS kad-DHT) can dial this node directly. If your Yundera
|
||||
host is behind a strict firewall, open TCP/4001 inbound — without it, this
|
||||
peer can still *find* others but won't be findable by them.
|
||||
|
||||
**Cohort scoping (optional).** Out of the box, MetaShare joins the global
|
||||
`metamesh-share-default` cohort. To run a private group of archivists,
|
||||
change `KAD_NAMESPACE` to a shared secret string on every node in the
|
||||
group — only nodes with the matching namespace will see each other.
|
||||
|
||||
**POC status.** v0.2 has an in-memory record store (rebuilt from MetaCore
|
||||
on every restart, ~2.5s for ~500 records) and no announcement signing yet.
|
||||
Don't rely on this for production federation until v0.2.1.
|
||||
|
||||
The dashboard is protected by Yundera's built-in Authelia via OIDC — sign
|
||||
in with your existing Yundera account. The peer-to-peer API surface
|
||||
(`/api/peers`, `/api/peer/*`, `/api/file/*`, `/api/search*`) is intentionally
|
||||
bypassed so other peers can reach it without an OIDC session.
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metacore/meta-core
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metashare/config
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metashare/data
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metashare-${domain}
|
||||
webui-path: "/"
|
||||
|
After Width: | Height: | Size: 133 KiB |
@@ -0,0 +1,64 @@
|
||||
# MetaShare — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metashare-app` runs `user: 0:0`.
|
||||
2. It publishes host port `4001/tcp` (libp2p).
|
||||
3. It mounts `/DATA/AppData/metacore/meta-core` — another app's AppData directory.
|
||||
4. Six HTTP paths bypass the SSO gate and are proxied straight to the backend:
|
||||
`/health`, `/api/peers*`, `/api/peer/*`, `/api/file/*`, `/api/search*` and
|
||||
`/api/services`.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. MetaShare writes its service registration into `/meta-core/services/`, a directory
|
||||
created and owned by MetaCore's root-running container.
|
||||
2. libp2p is **not HTTP**. Caddy is an HTTP reverse proxy and cannot carry it, and remote
|
||||
peers resolve this node's public multiaddr from the kad-DHT and dial it directly. Without
|
||||
a published, inbound-reachable TCP port the node can only make outbound connections and
|
||||
never serves bytes to the swarm. The app therefore carries the reserved
|
||||
`needs-public-ip` tag so a user behind CGNAT or the mesh-router tunnel can see the
|
||||
limitation before installing.
|
||||
3. `/meta-core` is the shared leader-election and service-discovery directory. MetaCore is
|
||||
a hard prerequisite of this app.
|
||||
4. Those are the **federation surface**: remote MetaShare peers call them to enumerate
|
||||
peers, resolve records, search, and fetch blocks. A remote peer is a machine client with
|
||||
no cookie jar and no way to complete an interactive OIDC redirect, so gating them makes
|
||||
the app unable to federate at all — which is its only function. Every other path,
|
||||
including the whole dashboard, stays behind the gate.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`),
|
||||
which gates every path behind the PCS's Authelia SSO over OIDC. The backend carries no
|
||||
Caddy labels and publishes no HTTP port to the host, so it is reachable only from the
|
||||
`pcs` Docker network.
|
||||
- Every service declares `cpu_shares`, and the memory-heavy ones a hard `mem_limit` with
|
||||
`memswap_limit` equal to it (swap disabled, so the container is OOM-killed rather than
|
||||
pushing the host into swap).
|
||||
- Image tags are pinned; no `:latest` anywhere in the stack.
|
||||
|
||||
- The bypassed paths are read-only federation endpoints scoped to content the user has
|
||||
explicitly shared. They expose no configuration, no dashboard and no write operation, and
|
||||
they serve exactly what this node already publishes to the public swarm over libp2p — so
|
||||
gating them would protect nothing that the DHT does not already carry.
|
||||
- The published port speaks libp2p only. libp2p transport is authenticated and encrypted
|
||||
(Noise) and peer identity is a public key, so an unauthenticated dial cannot read
|
||||
anything the node has not chosen to publish to the swarm. The HTTP API stays on
|
||||
`expose:` behind AppShield.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **Proxy libp2p through Caddy.** Not possible: Caddy fronts HTTP, and the DHT advertises a
|
||||
raw TCP multiaddr.
|
||||
- **Outbound-only / relayed transport.** Works for reachability but not for serving: a node
|
||||
nobody can dial contributes no bytes, which is the app's entire function.
|
||||
- **Run as `$PUID:$PGID`.** Fails writing into MetaCore's root-owned
|
||||
`/meta-core/services/`.
|
||||
|
||||
## Data protection
|
||||
|
||||
MetaShare's own store lives under `/DATA/AppData/metashare/{config,data}/`, declared in
|
||||
`x-compose-app.folders` and archived by Maison on uninstall. Only content the user has
|
||||
explicitly shared is published to the swarm; nothing under `/DATA/Documents`,
|
||||
`/DATA/Downloads` or `/DATA/Media` is mounted into this container at all.
|
||||
|
After Width: | Height: | Size: 76 KiB |
|
After Width: | Height: | Size: 55 KiB |
|
After Width: | Height: | Size: 143 KiB |
|
After Width: | Height: | Size: 133 KiB |
@@ -0,0 +1,134 @@
|
||||
# MetaSort - MetaMesh Ecosystem
|
||||
|
||||
MetaSort is part of the **MetaMesh ecosystem**, a suite of 4 interconnected apps that work together to organize and serve your media library.
|
||||
|
||||
## The MetaMesh Ecosystem
|
||||
|
||||
| App | Role | Port | Dependency |
|
||||
|-----|------|------|------------|
|
||||
| **MetaCore** | Leader election, Redis, WebDAV, Authelia SSO | 80, 9091 | None (install first) |
|
||||
| **MetaSort** | Metadata extraction, file indexing, plugin system | 80 | Requires MetaCore |
|
||||
| **MetaFuse** | Virtual filesystem (FUSE + WebDAV) | 80 | Requires MetaCore |
|
||||
| **MetaStremio** | Stremio addon with HLS transcoding | 7000 | Requires MetaCore |
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ User's Media Files │
|
||||
│ /DATA/Downloads, /DATA/Media │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│ (read-only bind mounts at /files/watch/*)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ MetaCore │
|
||||
│ • Leader election (flock) │
|
||||
│ • Redis (metadata storage) │
|
||||
│ • Authelia OIDC (single sign-on for all dashboards) │
|
||||
│ • WebDAV file serving + dynamic mount management │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│
|
||||
┌───────────────┼───────────────┐
|
||||
▼ ▼ ▼
|
||||
/DATA/AppData/metacore/meta-core pcs (Yundera platform network)
|
||||
(bind mount) (inter-container comms)
|
||||
│ │
|
||||
┌─────┴─────┐ ┌─────────────┐ ┌─────────────┐
|
||||
▼ ▼ ▼ ▼ ▼ ▼
|
||||
┌─────────┐ ┌─────────┐ ┌─────────────┐
|
||||
│MetaSort │ │MetaFuse │ │ MetaStremio │
|
||||
└─────────┘ └─────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
## Shared Infrastructure
|
||||
|
||||
### MetaCore Directory (`/DATA/AppData/metacore/meta-core`)
|
||||
|
||||
A **bind mount** shared by all MetaMesh apps. Contains:
|
||||
|
||||
```
|
||||
/DATA/AppData/metacore/meta-core/
|
||||
├── db/redis/ # Redis database (metadata storage)
|
||||
├── locks/ # Leader election lock files (kv-leader.info)
|
||||
└── services/ # Service discovery JSON files
|
||||
```
|
||||
|
||||
**Why a bind mount?**
|
||||
- Persists independently of Docker volume lifecycle
|
||||
- Predictable location for debugging/backup
|
||||
- Survives app reinstalls
|
||||
|
||||
### User media directories — only mounted into MetaCore
|
||||
|
||||
User media is bind-mounted **only into the MetaCore container**:
|
||||
|
||||
```
|
||||
/files/ (inside MetaCore container)
|
||||
└── watch/
|
||||
├── downloads/ # ← /DATA/Downloads (host bind, read-only)
|
||||
└── media/ # ← /DATA/Media (host bind, read-only)
|
||||
```
|
||||
|
||||
**MetaSort, MetaFuse, and MetaStremio do not mount `/files` at all.** They reach files through MetaCore's WebDAV endpoint (`http://metacore/webdav/*`), discovering the URL at runtime from the leader-info file in `/meta-core/locks/kv-leader.info`.
|
||||
|
||||
This matches the dev stack (Architecture V3) — MetaCore is the single source of truth for media files; siblings are stateless w.r.t. file storage.
|
||||
|
||||
Earlier versions had two problems here:
|
||||
1. A `metamesh-files` named volume mounted at `/files` with the user's media bind-mounted as subpaths. `docker volume rm metamesh-files` then recursively traversed the bind-mount targets (which shared inodes with the host paths) and **deleted everything in `/DATA/Downloads` and `/DATA/Media`**.
|
||||
2. Even after dropping the named volume, every sibling app duplicated the bind mounts of `/DATA/Downloads` and `/DATA/Media` — multiplying the blast radius of any future bind-mount footgun and breaking the "MetaCore owns files" architectural promise.
|
||||
|
||||
Mounting media only into MetaCore eliminates both issues.
|
||||
|
||||
### Network (`pcs`)
|
||||
|
||||
All MetaMesh containers attach to the Yundera platform network `pcs` (declared `external: true` everywhere). No app-private docker network — services discover each other by container hostname over `pcs`.
|
||||
|
||||
## Installation Order
|
||||
|
||||
**MetaCore must be installed first** — it provisions the Authelia secrets and OIDC clients that the other apps depend on.
|
||||
|
||||
1. **Install MetaCore** → Creates Authelia secrets and OIDC clients (`metasort`, `metafuse`, `metacore`)
|
||||
2. **Install MetaSort, MetaFuse, MetaStremio** in any order
|
||||
|
||||
Each app's `pre-install-cmd` checks for `/DATA/AppData/metacore/authelia/secrets/client-<app>.plain` and aborts with an error if MetaCore hasn't been installed.
|
||||
|
||||
## Volume Permissions
|
||||
|
||||
| Resource | MetaCore | MetaSort | MetaFuse | MetaStremio |
|
||||
|----------|----------|----------|----------|-------------|
|
||||
| `/DATA/AppData/metacore/meta-core` | rw | rw | rw | rw |
|
||||
| `/DATA/Downloads`, `/DATA/Media` | ro (mounts at `/files/watch/*`) | — (WebDAV) | — (WebDAV) | — (WebDAV) |
|
||||
| `/DATA/AppData/<app>/cache` | rw (own) | rw (own) | — | rw (own) |
|
||||
| `/DATA/AppData/metafuse/config` | — | — | rw | — |
|
||||
| `/DATA/MetaFuse/Library` | — | — | rw (FUSE mount) | — |
|
||||
| `/DATA/AppData/metacore/authelia/secrets` | rw (creates) | ro (oauth sidecar) | ro (oauth sidecar) | — |
|
||||
| `pcs` network (Yundera platform) | external | external | external | external |
|
||||
|
||||
## Developer Notes
|
||||
|
||||
### Adding a New MetaMesh App
|
||||
|
||||
1. Add `pre-install-cmd` that checks for `/DATA/AppData/metacore/authelia/secrets/client-<app>.plain`
|
||||
2. Bind-mount `/DATA/AppData/metacore/meta-core:/meta-core:rw` for leader info / service registration
|
||||
3. **Do NOT bind-mount user media** — read files via MetaCore's WebDAV (`http://metacore/webdav/*`), URL discovered from `/meta-core/locks/kv-leader.info`
|
||||
4. Join the `pcs` network as `external: true` (Yundera platform-managed; do not declare an app-private network)
|
||||
5. Add an `oauth2-proxy` sidecar that reads its client secret from `/DATA/AppData/metacore/authelia/secrets/client-<app>.plain`
|
||||
6. Discover Redis at runtime by reading `/meta-core/locks/kv-leader.info`
|
||||
|
||||
### Debugging
|
||||
|
||||
```bash
|
||||
# Check Redis data (Redis runs in MetaCore)
|
||||
docker exec metacore redis-cli keys '*'
|
||||
|
||||
# Check leader election
|
||||
cat /DATA/AppData/metacore/meta-core/locks/kv-leader.info
|
||||
|
||||
# Check service discovery
|
||||
ls /DATA/AppData/metacore/meta-core/services/
|
||||
|
||||
# Check what's mounted into a service
|
||||
docker exec metasort ls -la /files/watch/
|
||||
```
|
||||
|
||||
### Testing Locally
|
||||
|
||||
The MetaMesh dev environment (`meta-root-v2/dev/`) mirrors this architecture: direct bind mounts of `${DATA_WATCH_PATH}`, `./cache/test/files`, and `./cache/meta-sort/plugin-output` at `/files/watch`, `/files/test`, `/files/plugin` — no parent named volume.
|
||||
@@ -0,0 +1,201 @@
|
||||
name: metasort
|
||||
|
||||
services:
|
||||
# Backend service. Reachable on the pcs network as `metasort-app`.
|
||||
# Public ingress goes through the `metasort` hash-lock proxy below.
|
||||
metasort-app:
|
||||
image: ghcr.io/worph/meta-sort:1.0.3
|
||||
container_name: metasort-app
|
||||
hostname: metasort-app
|
||||
restart: unless-stopped
|
||||
cpu_shares: 20
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
volumes:
|
||||
# Shared infrastructure state (leader election, Redis dump, service discovery)
|
||||
# Owned by MetaCore; mounted rw here for service registration + leader reads.
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metacore/meta-core:/meta-core:rw
|
||||
|
||||
# Application cache storage
|
||||
- ${DATA_ROOT:-/DATA}/AppData/metasort/cache:/data/cache
|
||||
|
||||
# Docker socket for container plugin management
|
||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||
|
||||
# Note: no /files mounts — files are accessed via WebDAV from MetaCore
|
||||
# (Architecture V3). MetaCore is the single source of truth for media files.
|
||||
|
||||
environment:
|
||||
- NODE_ENV=production
|
||||
- META_CORE_PATH=/meta-core
|
||||
- CACHE_FOLDER_PATH=/data/cache
|
||||
- SERVICE_NAME=meta-sort
|
||||
- SERVICE_VERSION=1.0.0
|
||||
- BASE_URL=https://metasort-$APP_DOMAIN
|
||||
- TMDB_API_KEY=
|
||||
- TMDB_LANGUAGE=en-US
|
||||
- TZ=$TZ
|
||||
# Container plugin configuration
|
||||
- CONTAINER_CALLBACK_URL=http://metasort-app/api/plugins/callback
|
||||
- CONTAINER_NETWORK=pcs
|
||||
|
||||
# Note: Runs as root for nginx, Redis, and file management
|
||||
user: 0:0
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
# OIDC perimeter for the MetaSort dashboard. Self-registers as an OIDC client
|
||||
# with the platform's auth-registrar on first login (no secrets to manage).
|
||||
#
|
||||
# Naming: container_name MUST equal the URL-prefix of the public hostname so
|
||||
# the registrar's redirect-URI allowlist accepts `metasort-<user>.<domain>`.
|
||||
metasort:
|
||||
image: ghcr.io/yundera/appshield:2.0.9
|
||||
cpu_shares: 80
|
||||
container_name: metasort
|
||||
hostname: metasort
|
||||
restart: unless-stopped
|
||||
user: "0:0"
|
||||
|
||||
expose:
|
||||
- 80
|
||||
|
||||
environment:
|
||||
OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
|
||||
BACKEND_HOST: "metasort-app"
|
||||
BACKEND_PORT: "80"
|
||||
LISTEN_PORT: "80"
|
||||
REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
|
||||
|
||||
depends_on:
|
||||
- metasort-app
|
||||
|
||||
labels:
|
||||
# Only /health bypasses hash-lock — internal cluster traffic (plugin
|
||||
# callbacks, /meta/*, /webdav/*, /file/cid) reaches metasort-app via
|
||||
# the pcs network and never traverses the public proxy, so no public
|
||||
# bypass is required for those paths.
|
||||
|
||||
# ---- metasort-${APP_DOMAIN} (gateway TLS) ----
|
||||
caddy_0: metasort-${APP_DOMAIN}
|
||||
caddy_0.import: gateway_tls
|
||||
caddy_0.0_handle: "/health"
|
||||
caddy_0.0_handle.reverse_proxy: "metasort-app:80"
|
||||
caddy_0.1_handle: ""
|
||||
caddy_0.1_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metasort-IP.nip.io (gateway TLS) ----
|
||||
caddy_1: metasort-${APP_PUBLIC_IP_DASH}.nip.io
|
||||
caddy_1.import: gateway_tls
|
||||
caddy_1.0_handle: "/health"
|
||||
caddy_1.0_handle.reverse_proxy: "metasort-app:80"
|
||||
caddy_1.1_handle: ""
|
||||
caddy_1.1_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
# ---- metasort-IP.sslip.io (Let's Encrypt) ----
|
||||
caddy_2: metasort-${APP_PUBLIC_IP_DASH}.sslip.io
|
||||
caddy_2.0_handle: "/health"
|
||||
caddy_2.0_handle.reverse_proxy: "metasort-app:80"
|
||||
caddy_2.1_handle: ""
|
||||
caddy_2.1_handle.reverse_proxy: "{{upstreams 80}}"
|
||||
|
||||
networks:
|
||||
- pcs
|
||||
|
||||
networks:
|
||||
pcs:
|
||||
name: ${APP_NET:-pcs}
|
||||
external: true
|
||||
|
||||
x-casaos:
|
||||
architectures:
|
||||
- amd64
|
||||
- arm64
|
||||
author: Worph Team
|
||||
category: Media
|
||||
developer: Aptero Team
|
||||
|
||||
description:
|
||||
en_us: |
|
||||
**Smart metadata extraction and file indexing for massive media libraries.**
|
||||
|
||||
MetaSort is the brain of the MetaMesh ecosystem. It watches your media folders, extracts metadata using intelligent plugins, and stores everything in a distributed database for other services to use.
|
||||
|
||||
**Key Features:**
|
||||
- **Plugin-Based Processing** - Containerized plugins for FFmpeg analysis, TMDB enrichment, filename parsing, and more
|
||||
- **Smart Metadata Extraction** - Automatically extracts titles, seasons, episodes, quality, language, codecs
|
||||
- **Duplicate Detection** - Finds exact duplicates and different quality versions of the same content
|
||||
- **Real-time Monitoring** - Watches folders and instantly processes new files
|
||||
- **Built for Scale** - Handles collections up to 900k+ files and 100TB+
|
||||
- **Leader Election** - Distributed architecture with automatic failover
|
||||
|
||||
**Requires:** MetaCore (install first)
|
||||
|
||||
**Works with:** MetaFuse (virtual filesystem), MetaStremio (streaming)
|
||||
|
||||
**Dashboard Features:** File monitoring, plugin status, duplicate detection, metadata browser, processing queue
|
||||
|
||||
fr_fr: |
|
||||
**Extraction intelligente de metadonnees et indexation de fichiers pour d'immenses bibliotheques multimedia.**
|
||||
|
||||
MetaSort est le cerveau de l'ecosysteme MetaMesh. Il surveille vos dossiers multimedia, extrait les metadonnees a l'aide de plugins intelligents et stocke tout dans une base de donnees distribuee.
|
||||
|
||||
**Fonctionnalites Cles:**
|
||||
- **Traitement par Plugins** - Plugins conteneurises pour analyse FFmpeg, enrichissement TMDB, parsing de noms de fichiers
|
||||
- **Extraction Intelligente** - Extrait automatiquement titres, saisons, episodes, qualite, langue, codecs
|
||||
- **Detection de Doublons** - Trouve les doublons exacts et differentes versions de qualite
|
||||
- **Surveillance Temps Reel** - Surveille les dossiers et traite instantanement les nouveaux fichiers
|
||||
- **Concu pour l'Echelle** - Gere des collections jusqu'a 900k+ fichiers et 100TB+
|
||||
|
||||
**Necessite:** MetaCore (installer en premier)
|
||||
|
||||
**Fonctionne avec:** MetaFuse (systeme de fichiers virtuel), MetaStremio (streaming)
|
||||
|
||||
tagline:
|
||||
en_us: Smart metadata extraction engine - Index and organize massive media libraries with intelligent plugins
|
||||
fr_fr: Moteur d'extraction de metadonnees - Indexez et organisez d'immenses bibliotheques avec des plugins intelligents
|
||||
|
||||
title:
|
||||
en_us: MetaSort
|
||||
|
||||
icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaSort/icon.png
|
||||
screenshot_link:
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaSort/screenshot-1.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaSort/screenshot-2.png
|
||||
- https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaSort/screenshot-3.png
|
||||
thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaSort/thumbnail.png
|
||||
|
||||
index: /
|
||||
store_app_id: metasort
|
||||
main: metasort
|
||||
is_uncontrolled: false
|
||||
|
||||
tips:
|
||||
before_install:
|
||||
en_us: |
|
||||
**Requires MetaCore** - Install MetaCore first.
|
||||
|
||||
MetaSort connects to MetaCore for metadata storage. The dashboard is protected by
|
||||
Yundera's built-in Authelia via OIDC — sign in with your existing Yundera account.
|
||||
|
||||
# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
|
||||
# schema_version 2 is required for `folders`: an older Maison refuses the app
|
||||
# outright rather than starting it without its directories.
|
||||
x-compose-app:
|
||||
schema_version: 2
|
||||
# Created and chowned by Maison before any image is pulled, before any hook
|
||||
# runs and before the containers start — on the first boot and every one after.
|
||||
# Compose would otherwise create a missing bind source as a root-owned dir.
|
||||
folders:
|
||||
- path: /DATA/AppData/metacore/meta-core
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
- path: /DATA/AppData/metasort/cache
|
||||
user: $PUID
|
||||
group: $PGID
|
||||
# Same string as the caddy_0 label, so the tile link and the route agree.
|
||||
webui-host: metasort-${domain}
|
||||
webui-path: "/"
|
||||
|
After Width: | Height: | Size: 134 KiB |
@@ -0,0 +1,48 @@
|
||||
# MetaSort — Rationale
|
||||
|
||||
## What deviation / exception is being requested
|
||||
|
||||
1. `metasort-app` runs `user: 0:0`.
|
||||
2. It mounts the Docker socket, `/var/run/docker.sock`, read-only.
|
||||
3. It mounts `/DATA/AppData/metacore/meta-core` — another app's AppData directory.
|
||||
|
||||
## Why it is necessary
|
||||
|
||||
1. MetaSort writes its service registration into `/meta-core/services/`, a directory
|
||||
created and owned by MetaCore's root-running container, and runs nginx internally.
|
||||
2. MetaSort's enrichment plugins are **containers**. It lists, starts and stops them on the
|
||||
`pcs` network (`CONTAINER_NETWORK=pcs`), and receives their results on
|
||||
`CONTAINER_CALLBACK_URL`. Driving container lifecycle requires the Docker API.
|
||||
3. `/meta-core` is the shared leader-election and service-discovery directory. MetaCore is
|
||||
a hard prerequisite of this app.
|
||||
|
||||
## Security mitigations in place
|
||||
|
||||
- **The socket is mounted `:ro`.** That limits the *bind*, not the API — a read-only socket
|
||||
bind still permits write calls — so the meaningful control is the perimeter: the
|
||||
container carries no Caddy labels, publishes no host port, and is reachable only from
|
||||
the `pcs` network.
|
||||
- Public ingress terminates at the AppShield sidecar (`ghcr.io/yundera/appshield:2.0.9`)
|
||||
behind Authelia SSO; only `/health` is exempt.
|
||||
- MetaSort's own state is confined to `/DATA/AppData/metasort/cache/`, declared in
|
||||
`x-compose-app.folders`.
|
||||
- `cpu_shares` set on both services; image tags pinned.
|
||||
|
||||
## Alternatives considered and rejected
|
||||
|
||||
- **A Docker-socket proxy (e.g. tecnativa/docker-socket-proxy) allowing only the endpoints
|
||||
MetaSort uses.** This is the right long-term shape and is the intended follow-up. It is
|
||||
not shipped here because plugin lifecycle needs `containers`, `images`, `exec` and
|
||||
`networks` write access, which is most of the API surface anyway; the proxy would narrow
|
||||
the blast radius without eliminating it.
|
||||
- **Run plugins as sibling compose services instead of on-demand containers.** Plugins are
|
||||
installed and removed by the user at runtime; a static compose file cannot express that.
|
||||
- **Run as `$PUID:$PGID`.** Fails writing into MetaCore's root-owned
|
||||
`/meta-core/services/`.
|
||||
|
||||
## Data protection
|
||||
|
||||
MetaSort holds no user media — it reads metadata through MetaCore and caches derived data
|
||||
under `/DATA/AppData/metasort/cache/`, which Maison archives and restores. Its access to the
|
||||
user's files is indirect, through MetaCore's WebDAV endpoint, and is bounded by what
|
||||
MetaCore itself exposes (read-only mounts of `/DATA/Downloads` and `/DATA/Media`).
|
||||
|
After Width: | Height: | Size: 67 KiB |
|
After Width: | Height: | Size: 126 KiB |
|
After Width: | Height: | Size: 106 KiB |