metastar/meta-stremio
4
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

improve doc + bugfix

Browse Source
This commit is contained in:
Pierre
2026-05-25 23:48:20 +02:00
parent 37223c442b
commit 6cf93b72a7
2 changed files with 164 additions and 179 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+148 -175
View File
@@ -1,16 +1,11 @@
# Meta-Stremio # Meta-Stremio
A standalone Stremio addon service for streaming personal media with on-the-fly HLS transcoding. Part of the MetaMesh decentralized media ecosystem. Stremio addon for the MetaMesh stack. Reads file metadata from the shared KV store, reads files via WebDAV (or direct mount), and serves HLS with on-the-fly FFmpeg transcoding plus the standard Stremio addon protocol.
External port in the dev stack: **8182** (auth-gated, via Caddy + nginx-hash-lock). Internal container port is `7000`. Debug-direct port (bypasses the perimeter): **18182**. Container name: `metastremio-app` (backend) / `metastremio` (hash-lock proxy).
## Overview ## Overview
Meta-Stremio is a **process-read** service that:
- Reads file metadata from the shared KV store (Redis)
- Reads media files via WebDAV (from meta-sort) or direct volume access
- Provides HLS streaming with adaptive real-time FFmpeg transcoding
- Serves as a Stremio addon for seamless playback
- Supports SMB/rclone mounts via meta-sort's WebDAV endpoint
``` ```
┌─────────────────────────────────────────────────────────────────────┐ ┌─────────────────────────────────────────────────────────────────────┐
│ MetaMesh Ecosystem │ │ MetaMesh Ecosystem │
@@ -24,245 +19,223 @@ Meta-Stremio is a **process-read** service that:
│ ▼ ▼ ▼ │ │ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────┐ │
│ │ Shared KV Store (Redis) │ │ │ │ Shared KV Store (Redis) │ │
│ │ Leader election via flock on shared filesystem │ │ │ │ meta-core is leader, exposes /urls + /meta APIs │ │
│ │ DB stored in: DATA/Apps/meta-core/DB │ │
│ └─────────────────────────────────────────────────────┘ │ │ └─────────────────────────────────────────────────────┘ │
│ │ │ │ │ │
│ ▼ │ │ ▼ │
│ ┌─────────────────────────────────────────────────────┐ │ │ ┌─────────────────────────────────────────────────────┐ │
│ │ Shared DATA Volume (media files) │ │ │ │ File access via WebDAV (or direct mount) │ │
│ │ meta-sort writes → meta-stremio/meta-fuse reads │ │
│ └─────────────────────────────────────────────────────┘ │ │ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────┘
``` ```
## Features ## Features
### High Performance - **HLS transcoding** — adaptive preset+CRF targeting a 60–80% transcode ratio, prefetch of N segments ahead, persistent segment cache.
- **Adaptive Quality** - Dynamic preset + CRF adjustment targeting 60-80% transcode ratio - **Stremio protocol** — manifest, catalog, meta, stream; one HLS stream per audio track; subtitles extracted to VTT.
- **Intelligent Prefetch** - 4 segments ahead for smooth playback - **Direct file serving** — range-request capable, for clients that can play the source directly.
- **Single-threaded Encode** - 100% CPU per segment for fastest encoding - **Storage abstractions** — `LeaderStorage` (auto-discovers Redis via meta-core), `RedisStorage` (direct URL), `direct` mode (reads from meta-core's HTTP API + SSE meta-stream).
- **Muxed Segments** - Video+audio in one FFmpeg call - **Path-token gate for addon URLs** — when `HASH_API_SEED` is set, addon paths must be prefixed with `/s/<token>`. Dashboard/browser paths are protected separately by the upstream hash-lock + OIDC sidecar (Stremio clients can't carry cookies, so they get the path-token instead).
- **Segment Caching** - Persistent cache for repeated access - **Language-configurable manifest** — `displayLanguage` in the per-install config (base64-in-path) selects which `titles` translation meta-sort's TMDB plugin populated.
### Stremio Integration
- **Catalog Endpoint** - Browse media library by type (movies, series)
- **Meta Endpoint** - Detailed video information with codec details
- **Stream Endpoint** - Multiple stream options per video
- **Multi-Audio** - Separate HLS streams per audio track
- **Subtitles** - Extract and serve embedded subtitles as VTT
- **Language Configuration** - Configure display language (uses metadata stored by meta-sort's TMDB plugin)
### Storage Abstraction
- **Leader Storage** - Auto-discovers meta-sort's Redis via leader election (recommended)
- **Redis Storage** - Direct Redis connection via URL
## Project Structure
```
meta-stremio/
├── README.md
├── CLAUDE.md # Development guide
├── Dockerfile
├── docker-compose.yml
├── requirements.txt # redis + requests
├── src/
│ ├── server.py # HTTP server + routing
│ ├── stremio.py # Stremio handlers + storage integration
│ ├── transcoder.py # HLS transcoding engine
│ ├── fileserver.py # File serving (direct or WebDAV)
│ ├── webdav_client.py # WebDAV client for meta-sort file access
│ └── storage/
│ ├── __init__.py
│ ├── provider.py # Abstract StorageProvider
│ ├── redis_storage.py # Direct Redis connection
│ ├── leader_storage.py # Leader-aware Redis connection
│ └── leader_discovery.py # Leader election discovery
└── www/
└── index.html # Dashboard
```
## Installation ## Installation
### Docker (Recommended) ### Docker (in the MetaMesh dev stack)
```bash ```bash
# Build cd dev
docker build -t meta-stremio . ./scripts/start.sh
```
# Run with leader discovery (recommended - auto-discovers meta-sort's Redis) This brings up `metastremio-app` on port 18182 (debug-direct) and the auth-gated entry on `https://metastremio-dev.localhost:8182`.
### Docker (standalone)
```bash
docker build -t meta-stremio packages/meta-stremio
# Leader mode (auto-discover Redis via meta-core)
docker run -d \ docker run -d \
-p 7000:7000 \ -p 8182:7000 \
-v /path/to/media:/files:ro \ -v /path/to/media:/files:ro \
-v /path/to/meta-core:/meta-core \ -v /path/to/meta-core:/meta-core \
meta-stremio meta-stremio
# Run with direct Redis connection # Direct Redis mode
docker run -d \ docker run -d \
-p 7000:7000 \ -p 8182:7000 \
-v /path/to/media:/files:ro \ -v /path/to/media:/files:ro \
-e STORAGE_MODE=redis \ -e STORAGE_MODE=redis \
-e REDIS_URL=redis://your-redis:6379 \ -e REDIS_URL=redis://your-redis:6379 \
meta-stremio meta-stremio
``` ```
### Docker Compose The container always listens on port `7000` internally — map it to whatever you want externally.
### Standalone (development)
```bash ```bash
# Run as part of the MetaMesh stack (uses leader discovery) # Requires Python 3.9+ and FFmpeg.
cd dev pip install -r requirements.txt # redis, watchdog, Pillow, requests
./start.sh
```
### Standalone (Development)
```bash
# Prerequisites: Python 3.9+, FFmpeg
# Install dependencies
pip install -r requirements.txt
# Run
cd src cd src
python server.py python server.py
``` ```
## Configuration ## Configuration
### Environment Variables ### Environment variables
| Variable | Default | Description | | Variable | Default | Description |
|----------|---------|-------------| |---|---|---|
| `PORT` | `7000` | Server listen port | | `PORT` | `7000` | Internal HTTP listen port (the dev stack maps 18182 / 8182 to this). |
| `MEDIA_DIR` | `/data/media` | Media files directory | | `BASE_URL` | — | External URL emitted in manifests / poster URLs (e.g. `https://metastremio-dev.localhost:8182`). |
| `CACHE_DIR` | `/data/cache` | Transcoded segment cache | | `MEDIA_DIR` | `/files/watch` | Media directory (consumed by `transcoder.py`). |
| `STORAGE_MODE` | `leader` | `leader` or `redis` | | `CACHE_DIR` | `/data/cache` | Transcoded segment cache. |
| `META_CORE_PATH` | `/meta-core` | Path to meta-core shared volume (for leader discovery) | | `STORAGE_MODE` | `leader` | `leader`, `redis`, or `direct`. |
| `FILES_PATH` | `/files` | Path to files volume | | `META_CORE_PATH` | `/meta-core` | Shared meta-core volume; used by `LeaderClient` for `kv-leader.info`. |
| `REDIS_URL` | `redis://localhost:6379` | Redis connection URL (for `redis` mode) | | `META_CORE_URL` | — | meta-core HTTP API (e.g. `http://metacore-app:9000`); required for `STORAGE_MODE=direct` since there is no leader file to read. |
| `REDIS_PREFIX` | `meta-sort:` | Redis key prefix | | `FILES_PATH` | `/files` | Files volume. |
| `SEGMENT_DURATION` | `4` | HLS segment length (seconds) | | `REDIS_URL` | `redis://localhost:6379` | Redis URL for `STORAGE_MODE=redis`. |
| `PREFETCH_SEGMENTS` | `4` | Segments to prefetch ahead | | `REDIS_PREFIX` | `meta-sort:` | Redis key prefix. |
| `SCHEME` | `auto` | URL scheme (`http`, `https`, `auto`) | | `SCHEME` | `auto` | URL scheme for generated URLs: `http`, `https`, or `auto`. |
| `META_CORE_WEBDAV_URL` | - | WebDAV URL for file access (e.g., `http://meta-core/webdav`) | | `SEGMENT_DURATION` | `4` | HLS segment length (s). |
| `PREFETCH_SEGMENTS` | `4` | Segments to prefetch ahead. |
| `META_CORE_WEBDAV_URL` | — | When set, files are read over HTTP from meta-core's WebDAV instead of the filesystem (lets meta-stremio reach SMB / rclone mounts that only exist inside meta-core). |
| `HASH_API_SEED` | — | When set, derives a 16-char path-token; all addon routes must be prefixed with `/s/<token>`. Dashboard routes bypass. |
**WebDAV File Access**: When `META_CORE_WEBDAV_URL` is set, meta-stremio reads files via HTTP from meta-core's WebDAV server instead of direct filesystem access. This enables access to SMB/rclone mounts that are dynamically mounted inside meta-core. ## API endpoints
## API Endpoints All routes below are matched in `src/server.py`. When `HASH_API_SEED` is set, **every addon path is prefixed with `/s/<token>`**; the dashboard paths in the first table are *not* prefixed (they're protected by hash-lock/OIDC upstream).
### Dashboard / browser
### Dashboard
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |---|---|---|
| `/` | GET | Setup dashboard | | `/` , `/index.html` | GET | Setup dashboard |
| `/health` | GET | Health check with storage status | | `/configure` | GET | Per-install language configuration page |
| `/health` | GET | Liveness + storage status |
### Dashboard API
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/stats` | GET | Library statistics | | `/api/stats` | GET | Library statistics |
| `/api/library` | GET | Full library list | | `/api/library` | GET | Full library list (videos + count) |
| `/api/services` | GET | Discovered MetaMesh services (proxied from `meta-core /services`) |
| `/api/languages` | GET | Languages selectable from the `/configure` page |
### Stremio addon protocol
`/manifest.json` and `/stremio/manifest.json` are aliases; same for catalog/meta/stream. Stremio per-install configuration is encoded as URL-safe base64 JSON between the `/s/<token>` prefix and the addon path (e.g. `/s/<token>/<configB64>/manifest.json`).
### Stremio Protocol
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |---|---|---|
| `/manifest.json` | GET | Addon manifest | | `/manifest.json` | GET, HEAD | Addon manifest |
| `/catalog/:type/:id.json` | GET | Video catalog | | `/stremio/manifest.json` | GET, HEAD | Alias of `/manifest.json` |
| `/catalog/:type/:id.json` | GET | Catalog (with optional `…/:extra.json`) |
| `/stremio/catalog/:type/:id.json` | GET | Alias |
| `/meta/:type/:id.json` | GET | Video metadata | | `/meta/:type/:id.json` | GET | Video metadata |
| `/stremio/meta/:type/:id.json` | GET | Alias |
| `/stream/:type/:id.json` | GET | Stream URLs | | `/stream/:type/:id.json` | GET | Stream URLs |
| `/stremio/stream/:type/:id.json` | GET | Alias |
### File serving
### Transcoding
| Endpoint | Method | Description | | Endpoint | Method | Description |
|----------|--------|-------------| |---|---|---|
| `/transcode/:path/master.m3u8` | GET | ABR master playlist | | `/file/{cid}` | GET, HEAD | Serve file by CID |
| `/transcode/:path/master_original_a0.m3u8` | GET | Original quality playlist | | `/file/{cid}/w{width}` | GET, HEAD | Serve resized image by CID (uses Pillow) |
| `/transcode/:path/stream_a0_720p.m3u8` | GET | Quality stream playlist | | `/poster/{cid}` | GET, HEAD | Legacy alias of `/file/{cid}` |
| `/transcode/:path/seg_a0_720p_0.ts` | GET | Video segment | | `/poster/{cid}/w{width}` | GET, HEAD | Legacy alias of `/file/{cid}/w{width}` |
| `/transcode/:path/subtitle_0.vtt` | GET | Subtitle track | | `/direct/{path}` | GET, HEAD | Direct file serving with HTTP `Range` support |
| `/direct/:path` | GET | Direct file serving |
| `/transcode/metrics` | GET | Transcoding metrics |
## KV Store Schema ### Transcoder
Meta-stremio reads from meta-sort's Redis using this key format: | Endpoint | Method | Description |
|---|---|---|
| `/transcode/{path}/master.m3u8` | GET, HEAD | ABR master playlist |
| `/transcode/{path}/master_{resolution}.m3u8` | GET, HEAD | Master playlist for a single resolution |
| `/transcode/{path}/master_a{audio}.m3u8` | GET, HEAD | Master playlist for a single audio track |
| `/transcode/{path}/master_{resolution}_a{audio}.m3u8` | GET, HEAD | Combined variant |
| `/transcode/{path}/stream_a{audio}_{resolution}.m3u8` | GET, HEAD | Per-variant stream playlist |
| `/transcode/{path}/seg_a{audio}_{resolution}_{n}.ts` | GET, HEAD | Video segment |
| `/transcode/{path}/subtitle_{idx}.m3u8` | GET, HEAD | Subtitle playlist |
| `/transcode/{path}/subtitle_{idx}.vtt` | GET, HEAD | Subtitle VTT |
| `/transcode/metrics` | GET | Transcoder metrics (includes `total_files`) |
| `/transcode/reset-metrics` | POST | Reset transcoder metrics |
## Project structure
``` ```
meta-sort:file:{hashId} → Hash containing all metadata meta-stremio/
- filePath → /media/movies/Movie.mkv ├── README.md
- title → Movie Title ├── Dockerfile
- type → movie|series|anime ├── docker-compose.yml
- year → 2024 ├── docker/ # container entry scripts
- season → 1 (for series) ├── requirements.txt # redis, watchdog, Pillow, requests
- episode → 1 (for series) ├── src/
- duration → 7200 (seconds) │ ├── server.py # HTTP server + routing (entry point)
- width → 1920 │ ├── stremio.py # Stremio handlers + library/language helpers
- height → 1080 │ ├── transcoder.py # HLS transcoding engine + metrics + segment/subtitle managers
- videoCodec → h264 │ ├── fileserver.py # serve_file(cid, width) — file/poster endpoints
- audioCodec → aac │ ├── poster.py # poster URL/CID helpers
- audioTracks → JSON string │ ├── webdav_client.py # WebDAV client (file_exists, stream_file, stream_range, get_file_size)
- subtitles → JSON string │ └── storage/
- imdbId → tt1234567 │ ├── __init__.py
- poster → URL │ ├── provider.py # StorageProvider + VideoMetadata abstract base
│ ├── redis_storage.py # Direct Redis backend
│ ├── leader_storage.py # Leader-aware Redis backend
│ ├── leader_client.py # Reads /meta-core/locks/kv-leader.info + calls meta-core /urls
│ ├── meta_consumer.py # SSE consumer from meta-core /meta stream
│ ├── meta_core_api_client.py # HTTP client for meta-core REST API
│ └── service_registration.py # Registers meta-stremio with meta-core /services
└── www/
├── index.html # Dashboard
└── configure.html # Language configuration page
``` ```
## Stream Types ## KV store schema
Each video provides multiple stream options: meta-stremio reads from the per-file Redis hash `/file/{cid}` populated by meta-sort and its plugins. Field semantics (`videoType`, `originalTitle`, `titles`, `season`, `episode`, `movieYear`, `cid_*`, etc.) are documented in the repo-root [`METADATA_KEYS.md`](../../METADATA_KEYS.md) — that is the single source of truth, and matches the `@metazla/meta-interface` types.
1. **Direct File** - Original file, no transcoding ## Stream types
- Best quality, may not play on all devices
2. **HLS Original** - Transcoded at source resolution For each video, the addon advertises multiple streams:
- H.264/AAC in HLS container
- One stream per audio track
3. **HLS ABR** - Adaptive bitrate with quality ladder 1. **Direct file** — original bytes, served via `/direct/...` with `Range` support. Best quality, may not play on every device.
- Multiple resolutions (1080p, 720p, 480p, 360p) 2. **HLS original** — transcoded at source resolution, H.264/AAC, one stream per audio track.
- Automatic quality switching 3. **HLS ABR** — adaptive ladder. Quality presets are baked into `transcoder.py`.
## Transcoding Details | Resolution | Video bitrate | Audio bitrate |
|---|---|---|
### Adaptive Quality System
The transcoder automatically adjusts encoding settings to maintain a 60-80% transcode ratio (segment encode time / segment duration):
- **Too Fast (<60%)**: Increase quality (lower CRF, slower preset)
- **Too Slow (>80%)**: Decrease quality (higher CRF, faster preset)
### Quality Ladder
| Resolution | Video Bitrate | Audio Bitrate |
|------------|---------------|---------------|
| 1080p | 5000 kbps | 192 kbps | | 1080p | 5000 kbps | 192 kbps |
| 720p | 3000 kbps | 128 kbps | | 720p | 3000 kbps | 128 kbps |
| 480p | 1500 kbps | 128 kbps | | 480p | 1500 kbps | 128 kbps |
| 360p | 800 kbps | 96 kbps | | 360p | 800 kbps | 96 kbps |
## Adding to Stremio ## Adding to Stremio
1. Open the dashboard at `http://localhost:7000` 1. Open the dashboard at `https://metastremio-dev.localhost:8182/` (or debug-direct `http://localhost:18182/`).
2. Click "Install in Stremio" or copy the manifest URL 2. Pick your display language on `/configure` if you want non-English titles.
3. In Stremio: Settings → Addons → Enter URL → Install 3. Use the **Install in Stremio** button (uses the `stremio://` protocol handler) or copy the manifest URL into Stremio: Settings → Addons → Add Addon.
## Development ## Debugging
```bash ```bash
# Run in development mode (requires meta-sort running with Redis) # Container logs
cd src docker compose -f dev/docker-compose.yml logs -f metastremio-app
python server.py
# Or with direct Redis URL # Live health
STORAGE_MODE=redis REDIS_URL=redis://localhost:6379 python server.py curl -k https://metastremio-dev.localhost:8182/health
curl http://localhost:18182/health # debug-direct (no auth, no TLS)
# View logs # Transcoder metrics
tail -f /data/cache/*.log curl http://localhost:18182/transcode/metrics
curl -X POST http://localhost:18182/transcode/reset-metrics
``` ```
## Related Projects Container lifecycle: do **not** `docker restart metastremio-app` to apply config — use the dev-stack reload script (`./scripts/reload-stremio.sh`) so supervisord restarts the right process inside the container.
- **[meta-sort](../meta-sort)** - File indexer (writes metadata to Redis) ## Related projects
- **[meta-fuse](../meta-fuse)** - Virtual filesystem (reads metadata)
- **[meta-orbit](../meta-orbit)** - P2P metadata sharing - **[meta-sort](../meta-sort)** — file indexer; writes the metadata meta-stremio reads.
- **[meta-fuse](../meta-fuse)** — virtual filesystem; reads the same KV store.
- **[meta-share](../meta-share)** — decentralised metadata sharing.
## License ## License
src/storage/leader_storage.py
+16 -4
View File
@@ -221,17 +221,29 @@ class LeaderStorage(StorageProvider):
self._disconnect_redis() self._disconnect_redis()
def is_connected(self) -> bool: def is_connected(self) -> bool:
"""Check if connected to the storage backend (Redis or HTTP API).""" """Check if connected to the storage backend (Redis or HTTP API).
HTTP-only mode auto-recovers from transient meta-core outages: if
the health probe fails we drop the connected flag but keep the
api_client so the *next* call probes again. The background
reconnect thread is only needed for Redis-mode where the
connection itself is sticky.
"""
with self._lock: with self._lock:
if not self._connected: # HTTP-only mode: probe meta-core's /health on every call. This
return False # is cheap (a single HTTP GET) and naturally recovers when
# HTTP-only mode: probe meta-core's /health. # meta-core comes back without needing a separate reconnect
# loop. The api_client object stays valid through transient
# failures — it's stateless aside from the configured URL.
if self._api_client is not None and self._client is None: if self._api_client is not None and self._client is None:
if self._api_client.health(): if self._api_client.health():
self._connected = True
return True return True
self._connected = False self._connected = False
return False return False
# Legacy Redis mode. # Legacy Redis mode.
if not self._connected:
return False
if not self._client: if not self._client:
return False return False
try: try: