Import the MetaMesh app family from Worph/AppStore

23 apps (MetaCore, the library/network/client tiers and 13 MetaFeeder*) move out of the general-purpose Worph/AppStore into their own store so the MetaMesh family releases on its own cadence. - asset URLs repointed to cdn.jsdelivr.net/gh/Worph/MetaAppStore@main - store scaffolding added (category/recommend/featured lists, licence, contributing, gitattributes) - psd-source/meta-icons-v2 icon generator follows the apps MetaMCP stays in Worph/AppStore: it is third-party metatool-ai software, not part of MetaMesh.
2026-09-13 16:18:29 +00:00
parent 998eba19f4
commit 39f82e1f96
145 changed files with 12964 additions and 1 deletions
@@ -0,0 +1,241 @@
+name: metacore
+
+services:
+  # Backend service. Reachable on the pcs network as `metacore-app`.
+  # Public ingress goes through the `metacore` hash-lock proxy below — this
+  # service has no public Caddy labels and no exposed host ports.
+  metacore-app:
+    image: ghcr.io/worph/meta-core:1.0.12
+    container_name: metacore-app
+    # Hard RAM cap, swap disabled (memswap_limit == mem_limit) → OOM
+    # instead of swap. Both values must move together or swap comes back.
+    #
+    # Raised 768m -> 1536m (2026-08-26): on watch.nsl.sh the Redis working set
+    # ran at 73-97% of 768m, i.e. one indexing burst away from an OOM kill that
+    # takes the record store (and every service reading it) down with it.
+    # 1536m puts steady state near 30%.
+    mem_limit: 1536m
+    memswap_limit: 1536m
+    hostname: metacore-app
+    restart: unless-stopped
+    privileged: true
+    cpu_shares: 50
+
+    expose:
+      - 80
+
+    volumes:
+      # Shared infrastructure state (leader election, Redis dump, service discovery).
+      # Lives inside /DATA/AppData/metacore/ so CasaOS archives it on uninstall.
+      - ${DATA_ROOT:-/DATA}/AppData/metacore/meta-core:/meta-core:rw
+
+      # User's media directories — bind-mounted directly (no parent named volume,
+      # so `docker volume rm` cannot recurse into these and wipe host data).
+      - ${DATA_ROOT:-/DATA}/Downloads:/files/watch/downloads:ro
+      - ${DATA_ROOT:-/DATA}/Media:/files/watch/media:ro
+
+      # Application cache storage
+      - ${DATA_ROOT:-/DATA}/AppData/metacore/cache:/data/cache
+
+      # Plugin output — gateway/feeder-published artwork (posters, backdrops)
+      # lands here via meta-core's WebDAV endpoint, and every record's
+      # `filePath` points into it. This MUST be a bind mount: without it the
+      # directory is the container's writable layer, so any `docker compose
+      # up -d` that recreates metacore-app silently deletes every poster while
+      # Redis keeps pointing at them. That is exactly what happened on
+      # watch.nsl.sh on 2026-08-03 (270 files → 0), and it is the reason a
+      # fraction of poster CIDs 404 with a healthy-looking record behind them.
+      # metacore-app is the only service that mounts /files; the gateway and
+      # feeders write here over HTTP, so this one mount covers all of them.
+      - ${DATA_ROOT:-/DATA}/AppData/metacore/plugin:/files/plugin:rw
+
+    environment:
+      - BASE_URL=https://metacore-$APP_DOMAIN
+      - TZ=$TZ
+
+    # Runs as root for nginx, Redis, rclone mounts
+    user: 0:0
+
+    healthcheck:
+      test: ["CMD", "curl", "-sf", "http://localhost/health"]
+      interval: 5s
+      timeout: 3s
+      retries: 30
+      start_period: 30s
+    networks:
+      - pcs
+
+  # OIDC perimeter for the MetaCore dashboard. Self-registers as an OIDC client
+  # with the platform's auth-registrar on first login (no secrets to manage).
+  #
+  # Naming convention (per appshield + auth-registrar): the proxy's
+  # container_name MUST equal the URL-prefix of the public hostname, because the
+  # registrar's redirect-URI allowlist is "<container_name>" or "<container_name>-*".
+  # Our public URL is `metacore-${APP_DOMAIN}`, so the proxy container is named
+  # `metacore` and the backend takes the `-app` suffix.
+  metacore:
+    image: ghcr.io/yundera/appshield:2.0.9
+    cpu_shares: 80
+    container_name: metacore
+    hostname: metacore
+    restart: unless-stopped
+    user: "0:0"
+
+    expose:
+      - 80
+
+    environment:
+      OIDC_REGISTRAR_URL: "http://auth-registrar:9092"
+      BACKEND_HOST: "metacore-app"
+      BACKEND_PORT: "80"
+      LISTEN_PORT: "80"
+      REDIRECT_HOST_SUFFIXES: "${APP_DOMAIN},${APP_PUBLIC_IP_DASH}.nip.io,${APP_PUBLIC_IP_DASH}.sslip.io"
+
+    depends_on:
+      - metacore-app
+
+    labels:
+      # Only /health bypasses hash-lock — inter-service traffic (/meta/*,
+      # /webdav/*, /file/cid) flows over the pcs network from sibling apps
+      # (metasort-app, metafuse-app, metastremio) directly to metacore-app
+      # and never traverses the public proxy.
+
+      # ---- metacore-${APP_DOMAIN} (gateway TLS) ----
+      caddy_0: metacore-${APP_DOMAIN}
+      caddy_0.import: gateway_tls
+      caddy_0.0_handle: "/health"
+      caddy_0.0_handle.reverse_proxy: "metacore-app:80"
+      caddy_0.1_handle: ""
+      caddy_0.1_handle.reverse_proxy: "{{upstreams 80}}"
+
+      # ---- metacore-IP.nip.io (gateway TLS) ----
+      caddy_1: metacore-${APP_PUBLIC_IP_DASH}.nip.io
+      caddy_1.import: gateway_tls
+      caddy_1.0_handle: "/health"
+      caddy_1.0_handle.reverse_proxy: "metacore-app:80"
+      caddy_1.1_handle: ""
+      caddy_1.1_handle.reverse_proxy: "{{upstreams 80}}"
+
+      # ---- metacore-IP.sslip.io (Let's Encrypt) ----
+      caddy_2: metacore-${APP_PUBLIC_IP_DASH}.sslip.io
+      caddy_2.0_handle: "/health"
+      caddy_2.0_handle.reverse_proxy: "metacore-app:80"
+      caddy_2.1_handle: ""
+      caddy_2.1_handle.reverse_proxy: "{{upstreams 80}}"
+
+    networks:
+      - pcs
+
+networks:
+  pcs:
+    name: ${APP_NET:-pcs}
+    external: true
+
+x-casaos:
+  architectures:
+    - amd64
+    - arm64
+  author: Worph Team
+  category: Media
+  developer: Aptero Team
+
+  description:
+    en_us: |
+      **The infrastructure foundation for the MetaMesh media management ecosystem.**
+
+      MetaCore provides the shared services that power all MetaMesh applications: distributed leader election, Redis database, WebDAV file serving, remote mount management, and service discovery.
+
+      **Key Features:**
+      - **Leader Election** - Automatic flock-based leader election with failover support
+      - **Redis Database** - Centralized metadata storage for all MetaMesh services
+      - **WebDAV File Serving** - Cached file access for all services via HTTP
+      - **Mount Management** - Add NFS, SMB, and rclone remote mounts from the dashboard
+      - **File Watcher** - Monitors folders and notifies services of new files
+      - **Metadata Editor** - Browse and edit file metadata stored in Redis
+      - **Service Discovery** - Automatic registration and discovery of MetaMesh services
+      - **OIDC Single Sign-On** - Dashboard protected by Yundera's built-in Authelia (auto-registered, no secrets to configure)
+
+      **Install this FIRST** before MetaSort, MetaFuse, or MetaStremio.
+
+      **Dashboard Features:** Service health, file watchers, mount management, metadata editor, KV browser
+
+    fr_fr: |
+      **La fondation d'infrastructure pour l'ecosysteme de gestion de medias MetaMesh.**
+
+      MetaCore fournit les services partages qui alimentent toutes les applications MetaMesh : election de leader distribuee, base de donnees Redis, service de fichiers WebDAV, gestion de montages distants et decouverte de services.
+
+      **Fonctionnalites Cles:**
+      - **Election de Leader** - Election automatique basee sur flock avec support de basculement
+      - **Base de Donnees Redis** - Stockage centralise des metadonnees pour tous les services
+      - **Service de Fichiers WebDAV** - Acces aux fichiers en cache via HTTP
+      - **Gestion des Montages** - Ajout de montages NFS, SMB et rclone depuis le tableau de bord
+      - **Surveillance de Fichiers** - Surveille les dossiers et notifie les services des nouveaux fichiers
+      - **Editeur de Metadonnees** - Parcourir et editer les metadonnees stockees dans Redis
+      - **Decouverte de Services** - Enregistrement et decouverte automatiques des services
+      - **SSO OIDC** - Tableau de bord protege par l'Authelia integre de Yundera (auto-enregistre, aucun secret a configurer)
+
+      **Installez ceci EN PREMIER** avant MetaSort, MetaFuse ou MetaStremio.
+
+  tagline:
+    en_us: MetaMesh infrastructure - Leader election, Redis, WebDAV, and service discovery
+    fr_fr: Infrastructure MetaMesh - Election de leader, Redis, WebDAV et decouverte de services
+
+  title:
+    en_us: MetaCore
+
+  icon: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/icon.png
+  screenshot_link:
+    - https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/screenshot-1.png
+    - https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/screenshot-2.png
+    - https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/screenshot-3.png
+  thumbnail: https://cdn.jsdelivr.net/gh/Worph/MetaAppStore@main/Apps/MetaCore/thumbnail.png
+
+  index: /
+  store_app_id: metacore
+  main: metacore
+  is_uncontrolled: false
+
+  tips:
+    before_install:
+      en_us: |
+        **MetaCore is the foundation for the MetaMesh ecosystem.**
+
+        Install this FIRST before MetaSort, MetaFuse, or MetaStremio.
+
+        **What MetaCore can see:** it mounts your **`/DATA/Downloads`** and **`/DATA/Media`**
+        folders **read-only** so it can watch them for new files and index what it finds.
+        It cannot modify, rename or delete anything in them. Everything it writes stays in
+        `/DATA/AppData/metacore/`. See `rationale.md` for the full breakdown.
+
+        **Authentication:** The dashboard is protected by Yundera's built-in Authelia via OIDC.
+        Sign in with your existing Yundera account when you open the dashboard for the first time.
+
+        **After installation:**
+        1. Open the dashboard at `https://metacore-$APP_DOMAIN/`
+        2. Sign in via the Authelia portal that appears
+        3. Install MetaSort, MetaFuse, or MetaStremio as needed
+
+# Maison (Yundera dashboard) configuration. Read in preference to x-casaos.
+# schema_version 2 is required for `folders`: an older Maison refuses the app
+# outright rather than starting it without its directories.
+x-compose-app:
+  schema_version: 2
+  # Created and chowned by Maison before any image is pulled, before any hook
+  # runs and before the containers start — on the first boot and every one after.
+  # Compose would otherwise create a missing bind source as a root-owned dir.
+  folders:
+    - path: /DATA/AppData/metacore/meta-core
+      user: $PUID
+      group: $PGID
+    - path: /DATA/AppData/metacore/cache
+      user: $PUID
+      group: $PGID
+    - path: /DATA/AppData/metacore/plugin
+      user: $PUID
+      group: $PGID
+    - path: /DATA/AppData/metacore/plugin/share
+      user: $PUID
+      group: $PGID
+  # Same string as the caddy_0 label, so the tile link and the route agree.
+  webui-host: metacore-${domain}
+  webui-path: "/"
@@ -0,0 +1,142 @@
+# =============================================================================
+# Authelia Configuration for MetaMesh (OIDC provider mode)
+#
+# Template — MetaCore/pre-install-cmd substitutes the placeholders below at
+# install time: APP_DOMAIN, APP_PUBLIC_IP_DASH, HMAC secret, RSA JWKS private
+# key, and one argon2id hash per OIDC client.
+#
+# Protected apps reach Authelia via OIDC (not forward_auth) so each app's
+# session cookie can be host-scoped to its own <app>-__AUTH_DOMAIN__ host.
+# =============================================================================
+
+# Server settings (plain HTTP - Caddy edge proxy handles TLS).
+# Buffers bumped to cope with large OIDC requests (long state + cookies).
+server:
+  address: 'tcp://0.0.0.0:9091'
+  buffers:
+    read: 16384
+    write: 16384
+
+# Logging
+log:
+  level: info
+
+# Authentication backend (file-based, single user)
+authentication_backend:
+  file:
+    path: /config/users_database.yml
+
+# Session configuration — scoped to Authelia's own portal only.
+# Host-exact scope: cookie is set on 'auth-<domain>' and authelia_url is
+# 'https://auth-<domain>' — exact host match, valid cookie scope.
+# Protected apps get their own host-scoped sessions via oauth2-proxy sidecars.
+session:
+  cookies:
+    - domain: 'auth-__AUTH_DOMAIN__'
+      authelia_url: 'https://auth-__AUTH_DOMAIN__'
+      name: 'authelia_session'
+      same_site: 'lax'
+      inactivity: '5m'
+      expiration: '1h'
+      remember_me: '1M'
+
+# Storage (SQLite for single-user deployment)
+storage:
+  local:
+    path: /config/db.sqlite
+
+# Access control
+access_control:
+  default_policy: one_factor
+
+# TOTP (for 2FA)
+totp:
+  issuer: 'MetaMesh'
+  period: 30
+  skew: 1
+
+# Filesystem notifier (no SMTP needed for single-user)
+notifier:
+  filesystem:
+    filename: /config/notifications.txt
+
+# OIDC Identity Provider — each MetaMesh app registers as a client and
+# delegates auth to Authelia via the standard Authorization Code flow.
+identity_providers:
+  oidc:
+    hmac_secret: '__OIDC_HMAC__'
+    jwks:
+      - key_id: 'metamesh-default'
+        algorithm: 'RS256'
+        use: 'sig'
+        key: |
+__OIDC_JWKS_KEY__
+    clients:
+      - client_id: 'metacore'
+        client_name: 'MetaCore Dashboard'
+        client_secret: '__CLIENT_METACORE_HASH__'
+        public: false
+        authorization_policy: 'one_factor'
+        consent_mode: 'implicit'
+        require_pkce: false
+        redirect_uris:
+          - 'https://metacore-__AUTH_DOMAIN__/oauth2/callback'
+          - 'https://metacore-__APP_PUBLIC_IP_DASH__.nip.io/oauth2/callback'
+          - 'https://metacore-__APP_PUBLIC_IP_DASH__.sslip.io/oauth2/callback'
+        scopes:
+          - 'openid'
+          - 'profile'
+          - 'email'
+          - 'groups'
+        response_types:
+          - 'code'
+        grant_types:
+          - 'authorization_code'
+          - 'refresh_token'
+        token_endpoint_auth_method: 'client_secret_basic'
+
+      - client_id: 'metasort'
+        client_name: 'MetaSort Dashboard'
+        client_secret: '__CLIENT_METASORT_HASH__'
+        public: false
+        authorization_policy: 'one_factor'
+        consent_mode: 'implicit'
+        require_pkce: false
+        redirect_uris:
+          - 'https://metasort-__AUTH_DOMAIN__/oauth2/callback'
+          - 'https://metasort-__APP_PUBLIC_IP_DASH__.nip.io/oauth2/callback'
+          - 'https://metasort-__APP_PUBLIC_IP_DASH__.sslip.io/oauth2/callback'
+        scopes:
+          - 'openid'
+          - 'profile'
+          - 'email'
+          - 'groups'
+        response_types:
+          - 'code'
+        grant_types:
+          - 'authorization_code'
+          - 'refresh_token'
+        token_endpoint_auth_method: 'client_secret_basic'
+
+      - client_id: 'metafuse'
+        client_name: 'MetaFuse Dashboard'
+        client_secret: '__CLIENT_METAFUSE_HASH__'
+        public: false
+        authorization_policy: 'one_factor'
+        consent_mode: 'implicit'
+        require_pkce: false
+        redirect_uris:
+          - 'https://metafuse-__AUTH_DOMAIN__/oauth2/callback'
+          - 'https://metafuse-__APP_PUBLIC_IP_DASH__.nip.io/oauth2/callback'
+          - 'https://metafuse-__APP_PUBLIC_IP_DASH__.sslip.io/oauth2/callback'
+        scopes:
+          - 'openid'
+          - 'profile'
+          - 'email'
+          - 'groups'
+        response_types:
+          - 'code'
+        grant_types:
+          - 'authorization_code'
+          - 'refresh_token'
+        token_endpoint_auth_method: 'client_secret_basic'
@@ -0,0 +1,10 @@
+# Authelia Users Database for MetaMesh
+# Generated by pre-install-cmd with hashed password
+users:
+  admin:
+    disabled: false
+    displayname: 'Admin'
+    password: '__PASSWORD_HASH__'
+    email: '[email protected]'
+    groups:
+      - admins
@@ -0,0 +1,54 @@
+# MetaCore — Rationale
+
+## What deviation / exception is being requested
+
+1. `metacore-app` runs `privileged: true` and `user: 0:0`.
+2. It mounts two broad slices of the user's data area, read-only:
+   `/DATA/Downloads` and `/DATA/Media`.
+3. `/health` bypasses the SSO gate and is proxied straight to the backend.
+
+## Why it is necessary
+
+1. MetaCore is the MetaMesh infrastructure container: it runs nginx, Redis and — the
+   load-bearing part — **rclone remote mounts** (NFS, SMB, S3, WebDAV) that the user adds
+   from the dashboard. Creating a mount namespace inside the container needs `SYS_ADMIN`
+   and `/dev/fuse`; the image also runs several supervised daemons that bind their own
+   sockets. Neither works from an unprivileged, uid-1000 process.
+2. MetaCore is the file watcher for the whole ecosystem. It has to see the directories a
+   user actually drops media into, which are exactly `/DATA/Downloads` and `/DATA/Media`.
+   There is no narrower path that satisfies the feature.
+3. Sibling MetaMesh apps and the PCS health probe need an unauthenticated liveness
+   endpoint; routing it to the backend keeps the tile's health independent of whether
+   Authelia is up.
+
+## Security mitigations in place
+
+- **Both user-data mounts are `:ro`.** MetaCore reads and indexes; it cannot write to,
+  rename or delete anything under `/DATA/Downloads` or `/DATA/Media`.
+- Everything MetaCore writes lives under `/DATA/AppData/metacore/`, declared in
+  `x-compose-app.folders` and owned `$PUID:$PGID`.
+- The dashboard itself is behind AppShield + Authelia SSO; only `/health` is exempt, and
+  it returns a liveness object with no user data in it.
+- `mem_limit`/`memswap_limit` are both 768 MB, so the container is OOM-killed rather than
+  driving the host into swap.
+
+## Alternatives considered and rejected
+
+- **Drop `privileged` for a `cap_add` list.** Tried: rclone's FUSE mounts need
+  `SYS_ADMIN` plus an unconfined AppArmor profile plus `/dev/fuse`, which is most of what
+  `privileged` grants, and the remaining supervised daemons still failed to start. The
+  reduced form was not meaningfully smaller, and it was harder to reason about.
+- **Run as `$PUID:$PGID`.** The mount syscalls fail outright.
+- **Mount narrower paths than `/DATA/Downloads` / `/DATA/Media`.** These *are* the user's
+  download and media roots; a subdirectory would silently miss most of their library.
+- **Put `/health` behind the gate.** The PCS probe has no Authelia session, so the tile
+  would show unhealthy whenever SSO was restarting.
+
+## Data protection
+
+User media is mounted read-only and never modified. All MetaCore state (Redis dump,
+leader-election files, plugin artwork, cache) is under `/DATA/AppData/metacore/`, which is
+the unit Maison archives on uninstall and restores from backup — so an uninstall/reinstall
+cycle with "keep user data" comes back with metadata, posters and mounts intact. The
+`plugin/` directory is a bind mount specifically so that recreating the container cannot
+take published artwork with it.