Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d35a64a744 | ||
|
|
9061c7a37d | ||
|
|
b100b3a77c | ||
|
|
8e50b85e08 | ||
|
|
a03aa35169 | ||
|
|
797cf98afd | ||
|
|
0f034d9e95 | ||
|
|
9e6c3859c0 |
@@ -0,0 +1,6 @@
|
|||||||
|
# FIX CRLF always when developer has not set
|
||||||
|
# Linux/Mac: git config --global core.autocrlf input
|
||||||
|
# Windows: git config --global core.autocrlf true
|
||||||
|
# Auto detect text files and perform LF normalization
|
||||||
|
* text=auto
|
||||||
|
* eol=lf
|
||||||
@@ -25,3 +25,4 @@ logs/
|
|||||||
|
|
||||||
# Runtime data
|
# Runtime data
|
||||||
data/
|
data/
|
||||||
|
docker/__pycache__/
|
||||||
|
|||||||
@@ -121,6 +121,7 @@ RUN chmod +x /usr/local/bin/meta-core
|
|||||||
COPY docker/nginx.conf /etc/nginx/nginx.conf
|
COPY docker/nginx.conf /etc/nginx/nginx.conf
|
||||||
COPY docker/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
|
COPY docker/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
|
||||||
COPY docker/wsgidav.yaml /app/docker/
|
COPY docker/wsgidav.yaml /app/docker/
|
||||||
|
COPY docker/webdav_token_controller.py /app/docker/
|
||||||
COPY docker/welcome.html /app/welcome/
|
COPY docker/welcome.html /app/welcome/
|
||||||
COPY docker/start-fuse-driver.sh /app/docker/
|
COPY docker/start-fuse-driver.sh /app/docker/
|
||||||
RUN chmod 644 /app/welcome/welcome.html && \
|
RUN chmod 644 /app/welcome/welcome.html && \
|
||||||
|
|||||||
@@ -84,7 +84,7 @@ Meta-Fuse is a read-only process in the MetaMesh ecosystem that:
|
|||||||
│ [3] meta-stremio (PROCESS-READ) │
|
│ [3] meta-stremio (PROCESS-READ) │
|
||||||
│ └─► Reads metadata from KV, streams media content │
|
│ └─► Reads metadata from KV, streams media content │
|
||||||
│ │
|
│ │
|
||||||
│ [4] meta-orbit (SHARING-READ-WRITE) │
|
│ [4] meta-share (SHARING-READ-WRITE) │
|
||||||
│ └─► P2P metadata sync across network │
|
│ └─► P2P metadata sync across network │
|
||||||
│ │
|
│ │
|
||||||
└────────────────────────────────────────────────────────────────────────────┘
|
└────────────────────────────────────────────────────────────────────────────┘
|
||||||
@@ -104,7 +104,7 @@ http://meta-core:9000
|
|||||||
|
|
||||||
/urls API Response (JSON):
|
/urls API Response (JSON):
|
||||||
{
|
{
|
||||||
"hostname": "meta-core-dev",
|
"hostname": "metacore-app",
|
||||||
"baseUrl": "http://localhost:8083",
|
"baseUrl": "http://localhost:8083",
|
||||||
"apiUrl": "http://meta-core:9000",
|
"apiUrl": "http://meta-core:9000",
|
||||||
"redisUrl": "redis://meta-core:6379",
|
"redisUrl": "redis://meta-core:6379",
|
||||||
@@ -606,10 +606,10 @@ journalctl -u meta-fuse-driver -f
|
|||||||
cat /meta-core/locks/kv-leader.info
|
cat /meta-core/locks/kv-leader.info
|
||||||
|
|
||||||
# Check /urls API
|
# Check /urls API
|
||||||
curl -k https://dev.localhost:8083/api/urls
|
curl -k https://metacore-dev.localhost:8083/api/urls
|
||||||
|
|
||||||
# Verify redis is running (on leader)
|
# Verify redis is running (on leader)
|
||||||
docker exec meta-core-dev redis-cli ping
|
docker exec metacore-app redis-cli ping
|
||||||
|
|
||||||
# Check meta-fuse logs
|
# Check meta-fuse logs
|
||||||
docker logs meta-fuse | grep "KV"
|
docker logs meta-fuse | grep "KV"
|
||||||
@@ -632,10 +632,10 @@ docker logs meta-fuse | grep "wsgidav"
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Check Redis has file data (flat key format)
|
# Check Redis has file data (flat key format)
|
||||||
docker exec meta-core-dev redis-cli keys "file:*" | head -20
|
docker exec metacore-app redis-cli keys "file:*" | head -20
|
||||||
|
|
||||||
# Check file index
|
# Check file index
|
||||||
docker exec meta-core-dev redis-cli scard "file:__index__"
|
docker exec metacore-app redis-cli scard "file:__index__"
|
||||||
|
|
||||||
# Verify source files exist
|
# Verify source files exist
|
||||||
ls -la /files/watch/
|
ls -la /files/watch/
|
||||||
@@ -644,7 +644,7 @@ ls -la /files/watch/
|
|||||||
curl http://localhost:3000/api/fuse/stats
|
curl http://localhost:3000/api/fuse/stats
|
||||||
|
|
||||||
# Check streaming state builder stats
|
# Check streaming state builder stats
|
||||||
docker logs meta-fuse-dev | grep "State builder"
|
docker logs metafuse-app | grep "State builder"
|
||||||
```
|
```
|
||||||
|
|
||||||
## Technology Stack
|
## Technology Stack
|
||||||
@@ -665,7 +665,7 @@ Meta-Fuse is designed to work seamlessly with other MetaMesh services:
|
|||||||
|
|
||||||
- **meta-sort**: Provides metadata in KV database
|
- **meta-sort**: Provides metadata in KV database
|
||||||
- **meta-stremio**: Can stream files exposed by meta-fuse
|
- **meta-stremio**: Can stream files exposed by meta-fuse
|
||||||
- **meta-orbit**: Syncs metadata across P2P network
|
- **meta-share**: Syncs metadata across P2P network
|
||||||
|
|
||||||
### Connecting to meta-sort
|
### Connecting to meta-sort
|
||||||
|
|
||||||
@@ -673,13 +673,13 @@ Ensure meta-sort is running and has processed files:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Check meta-sort processing status
|
# Check meta-sort processing status
|
||||||
curl -k https://dev.localhost:8180/api/processing/status
|
curl -k https://metasort-dev.localhost:8180/api/processing/status
|
||||||
|
|
||||||
# Verify Redis has metadata (via meta-core)
|
# Verify Redis has metadata (via meta-core)
|
||||||
docker exec meta-core-dev redis-cli scard "file:__index__"
|
docker exec metacore-app redis-cli scard "file:__index__"
|
||||||
|
|
||||||
# Check meta:events stream has events
|
# Check meta:events stream has events
|
||||||
docker exec meta-core-dev redis-cli xlen "meta:events"
|
docker exec metacore-app redis-cli xlen "meta:events"
|
||||||
```
|
```
|
||||||
|
|
||||||
Meta-fuse will automatically discover the KV database through the shared lock file at `/meta-core/locks/kv-leader.info`.
|
Meta-fuse will automatically discover the KV database through the shared lock file at `/meta-core/locks/kv-leader.info`.
|
||||||
|
|||||||
@@ -72,3 +72,6 @@ autorestart=true
|
|||||||
stdout_logfile=/var/log/supervisor/wsgidav.log
|
stdout_logfile=/var/log/supervisor/wsgidav.log
|
||||||
stderr_logfile=/var/log/supervisor/wsgidav_error.log
|
stderr_logfile=/var/log/supervisor/wsgidav_error.log
|
||||||
startsecs=5
|
startsecs=5
|
||||||
|
environment=
|
||||||
|
PYTHONPATH="/app/docker",
|
||||||
|
TOKEN_STORE_PATH="/meta-fuse/config/webdav-tokens.json"
|
||||||
|
|||||||
@@ -0,0 +1,71 @@
|
|||||||
|
"""
|
||||||
|
WsgiDAV custom DomainController for meta-fuse WebDAV access tokens.
|
||||||
|
|
||||||
|
Reads `${TOKEN_STORE_PATH}` (a JSON file written by meta-fuse's TokenStore.ts)
|
||||||
|
fresh on every basic-auth request — no caching, so token revocation from the
|
||||||
|
dashboard takes effect on the next request.
|
||||||
|
|
||||||
|
Username is ignored: the WebDAV password IS the token. We accept any
|
||||||
|
non-empty username so generic clients (Finder, Plex, davfs2) that demand both
|
||||||
|
fields keep working.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from wsgidav.dc.base_dc import BaseDomainController
|
||||||
|
|
||||||
|
# Match the prefix produced by TokenStore.ts.
|
||||||
|
TOKEN_PREFIX = "mfwd_"
|
||||||
|
TOKEN_STORE_DEFAULT = "/meta-fuse/config/webdav-tokens.json"
|
||||||
|
|
||||||
|
|
||||||
|
class WebdavTokenController(BaseDomainController):
|
||||||
|
"""
|
||||||
|
Auth provider backed by meta-fuse's per-device token store.
|
||||||
|
|
||||||
|
WsgiDAV calls `basic_auth_user(realm, user_name, password, environ)` for
|
||||||
|
every request that requires auth. We hash the candidate password with
|
||||||
|
sha256 and check it against the persisted records.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, wsgidav_app: Any, config: dict) -> None:
|
||||||
|
super().__init__(wsgidav_app, config)
|
||||||
|
self._store_path = os.environ.get("TOKEN_STORE_PATH", TOKEN_STORE_DEFAULT)
|
||||||
|
|
||||||
|
# --- BaseDomainController contract --------------------------------------
|
||||||
|
|
||||||
|
def get_domain_realm(self, path_info: str, environ: dict) -> str:
|
||||||
|
# Single realm — all paths require the same token.
|
||||||
|
return "metafuse-webdav"
|
||||||
|
|
||||||
|
def require_authentication(self, realm: str, environ: dict) -> bool:
|
||||||
|
return True
|
||||||
|
|
||||||
|
def supports_http_digest_auth(self) -> bool:
|
||||||
|
# Tokens are not stored in plaintext, so we cannot answer digest
|
||||||
|
# challenges — clients must use basic auth (over TLS).
|
||||||
|
return False
|
||||||
|
|
||||||
|
def basic_auth_user(
|
||||||
|
self, realm: str, user_name: str, password: str, environ: dict
|
||||||
|
) -> bool:
|
||||||
|
if not password or not password.startswith(TOKEN_PREFIX):
|
||||||
|
return False
|
||||||
|
|
||||||
|
try:
|
||||||
|
with open(self._store_path, "r", encoding="utf-8") as fh:
|
||||||
|
store = json.load(fh)
|
||||||
|
except FileNotFoundError:
|
||||||
|
return False
|
||||||
|
except (OSError, json.JSONDecodeError):
|
||||||
|
# Treat unreadable store as "no valid tokens" — fail closed.
|
||||||
|
return False
|
||||||
|
|
||||||
|
candidate = hashlib.sha256(password.encode("utf-8")).hexdigest()
|
||||||
|
for token in store.get("tokens", []):
|
||||||
|
if token.get("hash") == candidate:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
+8
-16
@@ -3,17 +3,14 @@
|
|||||||
host: 0.0.0.0
|
host: 0.0.0.0
|
||||||
port: 8080
|
port: 8080
|
||||||
|
|
||||||
# Simple user authentication
|
# Authentication is delegated to a custom DomainController that validates the
|
||||||
simple_dc:
|
# basic-auth password against meta-fuse's per-device token store. Any
|
||||||
user_mapping:
|
# non-empty username is accepted; the password IS the token (`mfwd_*`).
|
||||||
"/":
|
http_authenticator:
|
||||||
metamesh:
|
domain_controller: webdav_token_controller.WebdavTokenController
|
||||||
password: metamesh
|
accept_basic: true
|
||||||
roles: ["editor"]
|
accept_digest: false
|
||||||
"/webdav":
|
default_to_digest: false
|
||||||
metamesh:
|
|
||||||
password: metamesh
|
|
||||||
roles: ["editor"]
|
|
||||||
|
|
||||||
# Mount the virtual filesystem
|
# Mount the virtual filesystem
|
||||||
provider_mapping:
|
provider_mapping:
|
||||||
@@ -31,11 +28,6 @@ server: cheroot
|
|||||||
server_args:
|
server_args:
|
||||||
numthreads: 10
|
numthreads: 10
|
||||||
|
|
||||||
# Authentication settings
|
|
||||||
accept_basic: true
|
|
||||||
accept_digest: false
|
|
||||||
default_to_digest: false
|
|
||||||
|
|
||||||
# HTTP configuration
|
# HTTP configuration
|
||||||
add_header_MS_Author_Via: false
|
add_header_MS_Author_Via: false
|
||||||
|
|
||||||
|
|||||||
@@ -535,29 +535,29 @@ List all discovered MetaMesh services.
|
|||||||
"services": [
|
"services": [
|
||||||
{
|
{
|
||||||
"name": "meta-sort",
|
"name": "meta-sort",
|
||||||
"url": "http://meta-sort-dev",
|
"url": "http://metasort-app",
|
||||||
"api": "http://meta-sort-dev/api",
|
"api": "http://metasort-app/api",
|
||||||
"status": "healthy",
|
"status": "healthy",
|
||||||
"role": "writer"
|
"role": "writer"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "meta-fuse",
|
"name": "meta-fuse",
|
||||||
"url": "http://meta-fuse-dev",
|
"url": "http://metafuse-app",
|
||||||
"api": "http://meta-fuse-dev/api",
|
"api": "http://metafuse-app/api",
|
||||||
"status": "healthy",
|
"status": "healthy",
|
||||||
"role": "reader"
|
"role": "reader"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "meta-stremio",
|
"name": "meta-stremio",
|
||||||
"url": "http://meta-stremio-dev",
|
"url": "http://metastremio-app",
|
||||||
"api": "http://meta-stremio-dev/api",
|
"api": "http://metastremio-app/api",
|
||||||
"status": "healthy",
|
"status": "healthy",
|
||||||
"role": "reader"
|
"role": "reader"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"current": "meta-fuse",
|
"current": "meta-fuse",
|
||||||
"leader": {
|
"leader": {
|
||||||
"hostname": "meta-core-dev",
|
"hostname": "metacore-app",
|
||||||
"baseUrl": "http://localhost:8083",
|
"baseUrl": "http://localhost:8083",
|
||||||
"apiUrl": "http://meta-core:9000",
|
"apiUrl": "http://meta-core:9000",
|
||||||
"redisUrl": "redis://meta-core:6379",
|
"redisUrl": "redis://meta-core:6379",
|
||||||
|
|||||||
@@ -350,7 +350,7 @@ The `onFileComplete` callback is called when:
|
|||||||
```
|
```
|
||||||
[meta-fuse] Starting meta-fuse...
|
[meta-fuse] Starting meta-fuse...
|
||||||
[KVManager] Waiting for leader...
|
[KVManager] Waiting for leader...
|
||||||
[LeaderClient] Leader found: meta-core-dev at redis://meta-core:6379
|
[LeaderClient] Leader found: metacore-app at redis://meta-core:6379
|
||||||
[meta-fuse] KV Manager ready, initializing VFS with streaming bootstrap...
|
[meta-fuse] KV Manager ready, initializing VFS with streaming bootstrap...
|
||||||
[StreamingStateBuilder] Initialized with 12 VFS-relevant properties
|
[StreamingStateBuilder] Initialized with 12 VFS-relevant properties
|
||||||
[meta-fuse] Replaying meta:events stream from position 0...
|
[meta-fuse] Replaying meta:events stream from position 0...
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import type { RenamingConfig, RenamingRule, PreviewResponse } from '../vfs/types
|
|||||||
import { TEMPLATE_VARIABLES } from '../vfs/types/RenamingRuleTypes.js';
|
import { TEMPLATE_VARIABLES } from '../vfs/types/RenamingRuleTypes.js';
|
||||||
import { TemplateEngine } from '../vfs/template/TemplateEngine.js';
|
import { TemplateEngine } from '../vfs/template/TemplateEngine.js';
|
||||||
import { ConditionEvaluator } from '../vfs/template/ConditionEvaluator.js';
|
import { ConditionEvaluator } from '../vfs/template/ConditionEvaluator.js';
|
||||||
|
import { TokenStore } from '../webdav/TokenStore.js';
|
||||||
|
|
||||||
const logger = new Logger({ name: 'APIServer' });
|
const logger = new Logger({ name: 'APIServer' });
|
||||||
|
|
||||||
@@ -30,6 +31,7 @@ export class APIServer {
|
|||||||
private vfs: VirtualFileSystem;
|
private vfs: VirtualFileSystem;
|
||||||
private kvManager: KVManager | null = null;
|
private kvManager: KVManager | null = null;
|
||||||
private config: Required<APIServerConfig>;
|
private config: Required<APIServerConfig>;
|
||||||
|
private tokenStore: TokenStore;
|
||||||
|
|
||||||
constructor(vfs: VirtualFileSystem, config: APIServerConfig = {}, kvManager?: KVManager) {
|
constructor(vfs: VirtualFileSystem, config: APIServerConfig = {}, kvManager?: KVManager) {
|
||||||
this.vfs = vfs;
|
this.vfs = vfs;
|
||||||
@@ -38,6 +40,10 @@ export class APIServer {
|
|||||||
port: config.port ?? parseInt(process.env.API_PORT ?? '3000', 10),
|
port: config.port ?? parseInt(process.env.API_PORT ?? '3000', 10),
|
||||||
host: config.host ?? process.env.API_HOST ?? '0.0.0.0',
|
host: config.host ?? process.env.API_HOST ?? '0.0.0.0',
|
||||||
};
|
};
|
||||||
|
// CONFIG_DIR is the same dir ConfigStorage uses for renaming rules.
|
||||||
|
this.tokenStore = new TokenStore({
|
||||||
|
configDir: process.env.CONFIG_DIR ?? '/meta-fuse/config',
|
||||||
|
});
|
||||||
|
|
||||||
this.app = Fastify({
|
this.app = Fastify({
|
||||||
logger: false,
|
logger: false,
|
||||||
@@ -88,6 +94,43 @@ export class APIServer {
|
|||||||
// Service discovery — proxies to meta-core's /api/services, resolving the
|
// Service discovery — proxies to meta-core's /api/services, resolving the
|
||||||
// current leader's URL via LeaderClient (no hardcoded hostname).
|
// current leader's URL via LeaderClient (no hardcoded hostname).
|
||||||
this.app.get('/api/services', this.handleServices.bind(this));
|
this.app.get('/api/services', this.handleServices.bind(this));
|
||||||
|
|
||||||
|
// WebDAV access tokens (per-device basic-auth passwords for /webdav).
|
||||||
|
this.app.get('/api/webdav-tokens', this.handleListTokens.bind(this));
|
||||||
|
this.app.post('/api/webdav-tokens', this.handleCreateToken.bind(this));
|
||||||
|
this.app.delete('/api/webdav-tokens/:id', this.handleRevokeToken.bind(this));
|
||||||
|
}
|
||||||
|
|
||||||
|
private async handleListTokens(_req: FastifyRequest, reply: FastifyReply): Promise<void> {
|
||||||
|
reply.send({ tokens: this.tokenStore.list() });
|
||||||
|
}
|
||||||
|
|
||||||
|
private async handleCreateToken(
|
||||||
|
req: FastifyRequest<{ Body: { label?: string } }>,
|
||||||
|
reply: FastifyReply
|
||||||
|
): Promise<void> {
|
||||||
|
const label = (req.body?.label ?? '').toString();
|
||||||
|
if (label.length > 200) {
|
||||||
|
reply.status(400).send({ error: 'label too long (max 200 chars)' });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const { plaintext, view } = this.tokenStore.create(label);
|
||||||
|
// The plaintext is included exactly once in this response — the dashboard
|
||||||
|
// surfaces it to the user and then discards it. Subsequent GETs do not
|
||||||
|
// include it.
|
||||||
|
reply.status(201).send({ ...view, token: plaintext });
|
||||||
|
}
|
||||||
|
|
||||||
|
private async handleRevokeToken(
|
||||||
|
req: FastifyRequest<{ Params: { id: string } }>,
|
||||||
|
reply: FastifyReply
|
||||||
|
): Promise<void> {
|
||||||
|
const ok = this.tokenStore.revoke(req.params.id);
|
||||||
|
if (!ok) {
|
||||||
|
reply.status(404).send({ error: 'token not found' });
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
reply.status(204).send();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import 'dotenv/config';
|
|||||||
import { Logger } from 'tslog';
|
import { Logger } from 'tslog';
|
||||||
import { KVManager } from './kv/KVManager.js';
|
import { KVManager } from './kv/KVManager.js';
|
||||||
import { RedisClient, StreamMessage } from './kv/RedisClient.js';
|
import { RedisClient, StreamMessage } from './kv/RedisClient.js';
|
||||||
|
import { SSEEventClient, type SSEEvent } from './kv/SSEEventClient.js';
|
||||||
import { VirtualFileSystem } from './vfs/VirtualFileSystem.js';
|
import { VirtualFileSystem } from './vfs/VirtualFileSystem.js';
|
||||||
import { StreamingStateBuilder } from './vfs/StreamingStateBuilder.js';
|
import { StreamingStateBuilder } from './vfs/StreamingStateBuilder.js';
|
||||||
import { APIServer } from './api/APIServer.js';
|
import { APIServer } from './api/APIServer.js';
|
||||||
@@ -76,13 +77,16 @@ async function main(): Promise<void> {
|
|||||||
let vfs: VirtualFileSystem | null = null;
|
let vfs: VirtualFileSystem | null = null;
|
||||||
let stateBuilder: StreamingStateBuilder | null = null;
|
let stateBuilder: StreamingStateBuilder | null = null;
|
||||||
let apiServer: APIServer | null = null;
|
let apiServer: APIServer | null = null;
|
||||||
|
let sseClient: SSEEventClient | null = null;
|
||||||
|
|
||||||
// Graceful shutdown
|
// Graceful shutdown
|
||||||
const shutdown = async (signal: string): Promise<void> => {
|
const shutdown = async (signal: string): Promise<void> => {
|
||||||
logger.info(`Received ${signal}, shutting down...`);
|
logger.info(`Received ${signal}, shutting down...`);
|
||||||
|
|
||||||
try {
|
try {
|
||||||
// Stop stream consumer first
|
// Stop SSE consumer first
|
||||||
|
if (sseClient) await sseClient.stop();
|
||||||
|
// Stop legacy stream consumer if it ever ran
|
||||||
if (redisClient) {
|
if (redisClient) {
|
||||||
redisClient.stopStreamConsumer();
|
redisClient.stopStreamConsumer();
|
||||||
}
|
}
|
||||||
@@ -155,35 +159,52 @@ async function main(): Promise<void> {
|
|||||||
// This enables vfs.refresh() to replay the event stream
|
// This enables vfs.refresh() to replay the event stream
|
||||||
vfs.setStreamingStateBuilder(stateBuilder);
|
vfs.setStreamingStateBuilder(stateBuilder);
|
||||||
|
|
||||||
// Bootstrap: Replay meta:events stream from position 0
|
// Bootstrap + live stream via SSE on /api/events/meta.
|
||||||
// This rebuilds state from all historical events
|
//
|
||||||
logger.info(`Replaying ${EVENTS_STREAM} stream from position 0 (streaming bootstrap)...`);
|
// We persist the cursor; on a clean restart we resume from where
|
||||||
const bootstrapStart = Date.now();
|
// we left off. On a `gap` event (cursor trimmed out of retention)
|
||||||
|
// we fall back to replaying from the oldest available entry,
|
||||||
const lastId = await redisClient.replayStream(
|
// which is good enough to rebuild VFS state because every property
|
||||||
EVENTS_STREAM,
|
// change has been republished into the stream by meta-core.
|
||||||
'0',
|
const apiUrl = leaderInfo?.apiUrl;
|
||||||
async (message: StreamMessage) => {
|
if (!apiUrl) {
|
||||||
|
throw new Error('meta-core apiUrl not available from leader info; cannot start SSE consumer');
|
||||||
|
}
|
||||||
|
logger.info(`Starting SSE consumer for /api/events/meta (apiUrl=${apiUrl})...`);
|
||||||
|
sseClient = new SSEEventClient({
|
||||||
|
url: `${apiUrl.replace(/\/+$/, '')}/api/events/meta`,
|
||||||
|
// Deliberately NOT persisting the cursor. meta-fuse holds
|
||||||
|
// VFS state purely in memory, so each restart needs to
|
||||||
|
// rebuild from a full event replay. Setting cursorPath to
|
||||||
|
// null + initialCursor to "0-0" forces the gap-handling
|
||||||
|
// path which resumes from the oldest available stream
|
||||||
|
// entry — i.e. effectively a full bootstrap.
|
||||||
|
cursorPath: null,
|
||||||
|
logTag: '[meta-fuse SSE]',
|
||||||
|
initialCursor: '0-0',
|
||||||
|
onEvent: async (e: SSEEvent) => {
|
||||||
|
// Translate SSE event back into the StreamMessage shape
|
||||||
|
// StreamingStateBuilder expects.
|
||||||
|
const message: StreamMessage = {
|
||||||
|
id: e.id,
|
||||||
|
type: e.event as any,
|
||||||
|
key: e.data?.key,
|
||||||
|
path: e.data?.path,
|
||||||
|
size: e.data?.size,
|
||||||
|
midhash256: e.data?.midhash256,
|
||||||
|
oldPath: e.data?.oldPath,
|
||||||
|
watcherId: e.data?.watcherId,
|
||||||
|
timestamp: e.data?.timestamp ?? e.data?.ts,
|
||||||
|
} as StreamMessage;
|
||||||
await handleStreamingEvent(message, stateBuilder!);
|
await handleStreamingEvent(message, stateBuilder!);
|
||||||
},
|
},
|
||||||
100 // Batch size
|
onGap: (payload) => {
|
||||||
);
|
logger.warn(`SSE cursor ${payload.requested} trimmed; resuming from ${payload.resumeFrom}`);
|
||||||
|
|
||||||
const bootstrapTime = Date.now() - bootstrapStart;
|
|
||||||
const stats = vfs.getStats();
|
|
||||||
logger.info(`Streaming bootstrap complete: ${stats.fileCount} files in ${bootstrapTime}ms`);
|
|
||||||
|
|
||||||
// Start live stream consumer from where replay left off
|
|
||||||
logger.info(`Starting live stream consumer from ${lastId}...`);
|
|
||||||
redisClient.startSimpleStreamConsumer(
|
|
||||||
EVENTS_STREAM,
|
|
||||||
lastId,
|
|
||||||
async (message: StreamMessage) => {
|
|
||||||
await handleStreamingEvent(message, stateBuilder!);
|
|
||||||
},
|
},
|
||||||
5000 // 5 second block timeout
|
});
|
||||||
).catch(error => {
|
// Fire-and-forget; start() returns when stop() fires.
|
||||||
logger.error('Stream consumer error:', error);
|
sseClient.start().catch((error) => {
|
||||||
|
logger.error('SSE consumer terminated:', error);
|
||||||
});
|
});
|
||||||
|
|
||||||
// Initialize API server (with kvManager for service discovery)
|
// Initialize API server (with kvManager for service discovery)
|
||||||
|
|||||||
@@ -113,7 +113,7 @@ export class KVManager {
|
|||||||
try {
|
try {
|
||||||
const leaderInfo = await this.leaderClient.waitForLeader(30000);
|
const leaderInfo = await this.leaderClient.waitForLeader(30000);
|
||||||
logger.info(`Connecting to Redis at ${leaderInfo.redisUrl}...`);
|
logger.info(`Connecting to Redis at ${leaderInfo.redisUrl}...`);
|
||||||
await this.connectToRedis(leaderInfo.redisUrl);
|
await this.connectToRedis(leaderInfo.redisUrl, leaderInfo.apiUrl);
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
logger.error(`Failed to connect to leader: ${error.message}`);
|
logger.error(`Failed to connect to leader: ${error.message}`);
|
||||||
throw error;
|
throw error;
|
||||||
@@ -141,7 +141,7 @@ export class KVManager {
|
|||||||
try {
|
try {
|
||||||
const leaderInfo = await this.leaderClient.waitForLeader(30000);
|
const leaderInfo = await this.leaderClient.waitForLeader(30000);
|
||||||
logger.info(`Reconnecting to Redis at ${leaderInfo.redisUrl}...`);
|
logger.info(`Reconnecting to Redis at ${leaderInfo.redisUrl}...`);
|
||||||
await this.connectToRedis(leaderInfo.redisUrl);
|
await this.connectToRedis(leaderInfo.redisUrl, leaderInfo.apiUrl);
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
logger.error(`Failed to reconnect: ${error.message}`);
|
logger.error(`Failed to reconnect: ${error.message}`);
|
||||||
}
|
}
|
||||||
@@ -178,20 +178,28 @@ export class KVManager {
|
|||||||
/**
|
/**
|
||||||
* Connect to Redis
|
* Connect to Redis
|
||||||
*/
|
*/
|
||||||
private async connectToRedis(url: string): Promise<void> {
|
private async connectToRedis(url: string, apiUrl?: string): Promise<void> {
|
||||||
try {
|
try {
|
||||||
// Disconnect existing client if any
|
// Disconnect existing client if any
|
||||||
await this.disconnectRedis();
|
await this.disconnectRedis();
|
||||||
|
|
||||||
// Create new client
|
// Create new client. When apiUrl is supplied, reads route
|
||||||
|
// through meta-core HTTP (api-mediated-access PR C).
|
||||||
|
// When url is also empty, the client is HTTP-only and never
|
||||||
|
// opens a Redis socket (post-PR D state).
|
||||||
this.redisClient = new RedisClient({
|
this.redisClient = new RedisClient({
|
||||||
url,
|
url, // may be '' in PR D world
|
||||||
prefix: this.config.redisPrefix,
|
prefix: this.config.redisPrefix,
|
||||||
filesVolume: this.config.filesPath
|
filesVolume: this.config.filesPath,
|
||||||
|
metaCoreApiUrl: apiUrl,
|
||||||
});
|
});
|
||||||
|
|
||||||
await this.redisClient.connect();
|
await this.redisClient.connect();
|
||||||
|
if (url) {
|
||||||
logger.info('Connected to Redis');
|
logger.info('Connected to Redis');
|
||||||
|
} else {
|
||||||
|
logger.info('Storage client ready (HTTP-only mode, no Redis)');
|
||||||
|
}
|
||||||
|
|
||||||
// Notify ready
|
// Notify ready
|
||||||
this.notifyReady();
|
this.notifyReady();
|
||||||
|
|||||||
@@ -88,7 +88,24 @@ export class LeaderClient {
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
this.cachedUrls = await response.json() as URLsResponse;
|
const parsed = await response.json() as URLsResponse;
|
||||||
|
// api-mediated-access PR D: redisUrl is no longer published by
|
||||||
|
// meta-core. If it shows up, an older build snuck back in —
|
||||||
|
// fail loudly so the rollback can't go unnoticed. Set
|
||||||
|
// ALLOW_LEGACY_REDIS_URL=1 to downgrade to a warning during a
|
||||||
|
// deliberate temporary rollback.
|
||||||
|
if (parsed.redisUrl) {
|
||||||
|
const msg =
|
||||||
|
'meta-core still publishes redisUrl; direct Redis access ' +
|
||||||
|
'was retired by the api-mediated-access lockdown. Verify ' +
|
||||||
|
'meta-core version.';
|
||||||
|
if (process.env.ALLOW_LEGACY_REDIS_URL === '1') {
|
||||||
|
logger.warn('WARNING: ' + msg);
|
||||||
|
} else {
|
||||||
|
throw new Error(msg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.cachedUrls = parsed;
|
||||||
this.urlsCacheTime = now;
|
this.urlsCacheTime = now;
|
||||||
return this.cachedUrls;
|
return this.cachedUrls;
|
||||||
} catch (error: any) {
|
} catch (error: any) {
|
||||||
|
|||||||
@@ -0,0 +1,121 @@
|
|||||||
|
/**
|
||||||
|
* MetaCoreApiClient — HTTP read client against meta-core for meta-fuse.
|
||||||
|
*
|
||||||
|
* meta-fuse needs to read file metadata (property GETs, full-record fetch,
|
||||||
|
* hashId enumeration) as it builds the VFS. After the api-mediated-access
|
||||||
|
* lockdown lands these go through meta-core's /meta/{hash} family instead
|
||||||
|
* of direct Redis reads.
|
||||||
|
*
|
||||||
|
* Only the methods meta-fuse actually calls are implemented. See
|
||||||
|
* meta-core docs/api-mediated-access.md for the full surface.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export interface MetaCoreApiClientConfig {
|
||||||
|
apiUrl: string;
|
||||||
|
timeoutMs?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class MetaCoreApiClient {
|
||||||
|
private apiUrl: string;
|
||||||
|
private timeoutMs: number;
|
||||||
|
|
||||||
|
constructor(config: MetaCoreApiClientConfig) {
|
||||||
|
this.apiUrl = config.apiUrl.replace(/\/+$/, '');
|
||||||
|
this.timeoutMs = config.timeoutMs ?? 30000;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* GET /meta/{hash}/{prop}. Returns the raw string value or null on 404.
|
||||||
|
* meta-core sends text/plain for property reads.
|
||||||
|
*/
|
||||||
|
async getProperty(hashId: string, property: string): Promise<string | null> {
|
||||||
|
const path = `/meta/${encodeURIComponent(hashId)}/${this.encodePropertyPath(property)}`;
|
||||||
|
const url = `${this.apiUrl}${path}`;
|
||||||
|
const response = await fetch(url, {
|
||||||
|
method: 'GET',
|
||||||
|
signal: AbortSignal.timeout(this.timeoutMs),
|
||||||
|
headers: { Accept: 'text/plain' },
|
||||||
|
});
|
||||||
|
if (response.status === 404) return null;
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`[MetaCoreApiClient] GET ${path} → ${response.status} ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const text = await response.text();
|
||||||
|
return text === '' ? null : text;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* GET /meta/{hash}. Returns the flat property map or null on 404.
|
||||||
|
*/
|
||||||
|
async getMetadataFlat(hashId: string): Promise<Record<string, string> | null> {
|
||||||
|
const path = `/meta/${encodeURIComponent(hashId)}`;
|
||||||
|
const body = await this.fetchJsonAllowing404('GET', path);
|
||||||
|
if (body === null) return null;
|
||||||
|
const flat = (body as any)?.metadata;
|
||||||
|
if (!flat || typeof flat !== 'object') return null;
|
||||||
|
return flat as Record<string, string>;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* GET /meta. Enumerates every known hashId.
|
||||||
|
*/
|
||||||
|
async getAllHashIds(): Promise<string[]> {
|
||||||
|
const body = await this.fetchJson('GET', '/meta');
|
||||||
|
const ids = (body as any)?.hashIds;
|
||||||
|
return Array.isArray(ids) ? ids : [];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* GET /health on meta-core. Used as the connectivity probe.
|
||||||
|
*/
|
||||||
|
async health(): Promise<boolean> {
|
||||||
|
try {
|
||||||
|
const response = await fetch(`${this.apiUrl}/health`, {
|
||||||
|
method: 'GET',
|
||||||
|
signal: AbortSignal.timeout(Math.min(this.timeoutMs, 5000)),
|
||||||
|
headers: { Accept: 'application/json' },
|
||||||
|
});
|
||||||
|
return response.ok;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private encodePropertyPath(property: string): string {
|
||||||
|
return property
|
||||||
|
.split('/')
|
||||||
|
.map(segment => encodeURIComponent(segment))
|
||||||
|
.join('/');
|
||||||
|
}
|
||||||
|
|
||||||
|
private async fetchJson(method: string, path: string): Promise<unknown> {
|
||||||
|
const url = `${this.apiUrl}${path}`;
|
||||||
|
const response = await fetch(url, {
|
||||||
|
method,
|
||||||
|
signal: AbortSignal.timeout(this.timeoutMs),
|
||||||
|
headers: { Accept: 'application/json' },
|
||||||
|
});
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`[MetaCoreApiClient] ${method} ${path} → ${response.status} ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const text = await response.text();
|
||||||
|
if (!text) return null;
|
||||||
|
try { return JSON.parse(text); } catch { return text; }
|
||||||
|
}
|
||||||
|
|
||||||
|
private async fetchJsonAllowing404(method: string, path: string): Promise<unknown> {
|
||||||
|
const url = `${this.apiUrl}${path}`;
|
||||||
|
const response = await fetch(url, {
|
||||||
|
method,
|
||||||
|
signal: AbortSignal.timeout(this.timeoutMs),
|
||||||
|
headers: { Accept: 'application/json' },
|
||||||
|
});
|
||||||
|
if (response.status === 404) return null;
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(`[MetaCoreApiClient] ${method} ${path} → ${response.status} ${response.statusText}`);
|
||||||
|
}
|
||||||
|
const text = await response.text();
|
||||||
|
if (!text) return null;
|
||||||
|
try { return JSON.parse(text); } catch { return text; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -12,6 +12,7 @@ import { Redis } from 'ioredis';
|
|||||||
import { Logger } from 'tslog';
|
import { Logger } from 'tslog';
|
||||||
import * as os from 'os';
|
import * as os from 'os';
|
||||||
import type { IKVClient, KeyValuePair } from './IKVClient.js';
|
import type { IKVClient, KeyValuePair } from './IKVClient.js';
|
||||||
|
import { MetaCoreApiClient } from './MetaCoreApiClient.js';
|
||||||
|
|
||||||
const logger = new Logger({ name: 'RedisClient' });
|
const logger = new Logger({ name: 'RedisClient' });
|
||||||
|
|
||||||
@@ -74,7 +75,6 @@ export interface FileMetadata {
|
|||||||
|
|
||||||
// Title information (for folder organization)
|
// Title information (for folder organization)
|
||||||
title?: string;
|
title?: string;
|
||||||
titles?: { eng?: string; [key: string]: string | undefined };
|
|
||||||
originalTitle?: string;
|
originalTitle?: string;
|
||||||
fileName?: string;
|
fileName?: string;
|
||||||
|
|
||||||
@@ -104,15 +104,27 @@ export interface RedisClientConfig {
|
|||||||
prefix?: string;
|
prefix?: string;
|
||||||
filesVolume?: string;
|
filesVolume?: string;
|
||||||
reconnectInterval?: number;
|
reconnectInterval?: number;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* meta-core HTTP API base URL. When set, read methods (getProperty,
|
||||||
|
* getProperties, getMetadataFlat, getAllHashIds, health) route through
|
||||||
|
* the /meta/{hash} HTTP family instead of touching Redis directly.
|
||||||
|
*
|
||||||
|
* This is the meta-fuse half of the api-mediated-access lockdown (see
|
||||||
|
* meta-core docs/api-mediated-access.md). Reads stay on Redis when
|
||||||
|
* left unset so tests / one-off scripts still work.
|
||||||
|
*/
|
||||||
|
metaCoreApiUrl?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export class RedisClient implements Partial<IKVClient> {
|
export class RedisClient implements Partial<IKVClient> {
|
||||||
private client: Redis | null = null;
|
private client: Redis | null = null;
|
||||||
private subscriber: Redis | null = null;
|
private subscriber: Redis | null = null;
|
||||||
private config: Required<RedisClientConfig>;
|
private config: Required<Omit<RedisClientConfig, 'metaCoreApiUrl'>> & Pick<RedisClientConfig, 'metaCoreApiUrl'>;
|
||||||
private isConnected = false;
|
private isConnected = false;
|
||||||
private reconnectTimer: NodeJS.Timeout | null = null;
|
private reconnectTimer: NodeJS.Timeout | null = null;
|
||||||
private subscriptions: Map<string, (message: string) => void> = new Map();
|
private subscriptions: Map<string, (message: string) => void> = new Map();
|
||||||
|
private apiClient: MetaCoreApiClient | null = null;
|
||||||
|
|
||||||
// Stream consumer state
|
// Stream consumer state
|
||||||
private streamConsumerRunning = false;
|
private streamConsumerRunning = false;
|
||||||
@@ -125,20 +137,33 @@ export class RedisClient implements Partial<IKVClient> {
|
|||||||
prefix: config.prefix ?? process.env.REDIS_PREFIX ?? '',
|
prefix: config.prefix ?? process.env.REDIS_PREFIX ?? '',
|
||||||
filesVolume: config.filesVolume ?? process.env.FILES_VOLUME ?? '/files',
|
filesVolume: config.filesVolume ?? process.env.FILES_VOLUME ?? '/files',
|
||||||
reconnectInterval: config.reconnectInterval ?? 5000,
|
reconnectInterval: config.reconnectInterval ?? 5000,
|
||||||
|
metaCoreApiUrl: config.metaCoreApiUrl,
|
||||||
};
|
};
|
||||||
|
|
||||||
|
if (this.config.metaCoreApiUrl) {
|
||||||
|
this.apiClient = new MetaCoreApiClient({ apiUrl: this.config.metaCoreApiUrl });
|
||||||
|
logger.info(`Reads routed through meta-core API at ${this.config.metaCoreApiUrl}`);
|
||||||
|
}
|
||||||
|
|
||||||
// Unique consumer name for this instance
|
// Unique consumer name for this instance
|
||||||
this.consumerName = `${os.hostname()}-${process.pid}`;
|
this.consumerName = `${os.hostname()}-${process.pid}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Connect to Redis
|
* Connect to Redis. In HTTP-only mode (no URL + apiClient set) this is
|
||||||
|
* a no-op — reads route through MetaCoreApiClient and stream
|
||||||
|
* consumption is owned by SSEEventClient at the index.ts layer.
|
||||||
*/
|
*/
|
||||||
async connect(): Promise<void> {
|
async connect(): Promise<void> {
|
||||||
if (this.client && this.isConnected) {
|
if (this.client && this.isConnected) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (this.apiClient && !this.config.url) {
|
||||||
|
logger.info('HTTP-only mode — no direct Redis connection');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
this.client = new Redis(this.config.url, {
|
this.client = new Redis(this.config.url, {
|
||||||
retryStrategy: (times: number) => {
|
retryStrategy: (times: number) => {
|
||||||
@@ -236,9 +261,18 @@ export class RedisClient implements Partial<IKVClient> {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Check if connected
|
* Check if the metadata backend is reachable.
|
||||||
|
*
|
||||||
|
* In HTTP-only mode (api-mediated-access PR D) there's no Redis socket;
|
||||||
|
* reachability is signalled by having an apiClient configured. The
|
||||||
|
* health badge is keyed off this; reporting `false` there would make
|
||||||
|
* the UI permanently show "Disconnected" even though the service is
|
||||||
|
* working fine over HTTP.
|
||||||
*/
|
*/
|
||||||
get connected(): boolean {
|
get connected(): boolean {
|
||||||
|
if (this.apiClient && !this.config.url) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
return this.isConnected;
|
return this.isConnected;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -255,6 +289,15 @@ export class RedisClient implements Partial<IKVClient> {
|
|||||||
* @returns The property value or null if not found
|
* @returns The property value or null if not found
|
||||||
*/
|
*/
|
||||||
async getProperty(hashId: string, property: string): Promise<string | null> {
|
async getProperty(hashId: string, property: string): Promise<string | null> {
|
||||||
|
if (this.apiClient) {
|
||||||
|
try {
|
||||||
|
return await this.apiClient.getProperty(hashId, property);
|
||||||
|
} catch (error: any) {
|
||||||
|
logger.error(`HTTP getProperty failed for ${hashId}/${property}:`, error.message);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if (!this.client || !this.isConnected) {
|
if (!this.client || !this.isConnected) {
|
||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
@@ -579,12 +622,6 @@ export class RedisClient implements Partial<IKVClient> {
|
|||||||
mtime = isNaN(parsed) ? parseFloat(data.mtime) : parsed;
|
mtime = isNaN(parsed) ? parseFloat(data.mtime) : parsed;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Parse titles - can be nested object or simple string
|
|
||||||
let titles: { eng?: string; [key: string]: string | undefined } | undefined;
|
|
||||||
if (data['titles/eng']) {
|
|
||||||
titles = { eng: data['titles/eng'] };
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse season/episode (can be 0 for specials)
|
// Parse season/episode (can be 0 for specials)
|
||||||
const season = data.season !== undefined ? parseInt(data.season, 10) : undefined;
|
const season = data.season !== undefined ? parseInt(data.season, 10) : undefined;
|
||||||
const episode = data.episode !== undefined ? parseInt(data.episode, 10) : undefined;
|
const episode = data.episode !== undefined ? parseInt(data.episode, 10) : undefined;
|
||||||
@@ -603,12 +640,19 @@ export class RedisClient implements Partial<IKVClient> {
|
|||||||
mtime,
|
mtime,
|
||||||
ctime: data.ctime ? parseFloat(data.ctime) : mtime,
|
ctime: data.ctime ? parseFloat(data.ctime) : mtime,
|
||||||
|
|
||||||
// Identification
|
// Identification.
|
||||||
hashId: hashId ?? data.cid_midhash256 ?? data.hashId,
|
//
|
||||||
|
// `hashId` is always supplied by the caller — every `parseMetadata`
|
||||||
|
// call site derives it from the Redis key (`file:<hashId>/…`), so the
|
||||||
|
// fallbacks below it were already unreachable. The two that used to sit
|
||||||
|
// here (`data.cid_midhash256`, then `data.hashId`) were also the wrong
|
||||||
|
// shape: `cid_*` fields no longer exist on records, and meta-core
|
||||||
|
// rejects them at the write boundary. Kept as a single defensive `??`
|
||||||
|
// rather than a chain of dead ones. See METADATA_KEYS.md §14.13.
|
||||||
|
hashId: hashId ?? data.hashId,
|
||||||
|
|
||||||
// Title information
|
// Title information
|
||||||
title: data.title,
|
title: data.title,
|
||||||
titles,
|
|
||||||
originalTitle: data.originalTitle,
|
originalTitle: data.originalTitle,
|
||||||
fileName: data.fileName,
|
fileName: data.fileName,
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,229 @@
|
|||||||
|
/**
|
||||||
|
* SSEEventClient — HTTP Server-Sent Events client for meta-core event streams.
|
||||||
|
*
|
||||||
|
* Replaces direct Redis Streams consumption. See the same-named module in
|
||||||
|
* meta-sort for the design rationale (meta-core docs/api-mediated-access.md).
|
||||||
|
*/
|
||||||
|
import { promises as fs } from 'fs';
|
||||||
|
import { dirname } from 'path';
|
||||||
|
|
||||||
|
export interface SSEEvent {
|
||||||
|
id: string;
|
||||||
|
event: string;
|
||||||
|
data: any;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface SSEClientOptions {
|
||||||
|
url: string;
|
||||||
|
cursorPath: string | null;
|
||||||
|
onEvent: (e: SSEEvent) => Promise<void> | void;
|
||||||
|
onGap?: (payload: { requested: string; resumeFrom: string; reason: string }) => Promise<void> | void;
|
||||||
|
logTag?: string;
|
||||||
|
reconnectBaseMs?: number;
|
||||||
|
reconnectMaxMs?: number;
|
||||||
|
flushIntervalMs?: number;
|
||||||
|
/** When set, sent as Last-Event-ID on the *first* connect (e.g. to
|
||||||
|
* force replay from a specific point). After the first event the
|
||||||
|
* persisted cursor takes over. */
|
||||||
|
initialCursor?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class SSEEventClient {
|
||||||
|
private readonly opts: Required<Omit<SSEClientOptions, 'onGap' | 'initialCursor'>> & Pick<SSEClientOptions, 'onGap' | 'initialCursor'>;
|
||||||
|
private cursor: string = '';
|
||||||
|
private cursorDirty = false;
|
||||||
|
private abort: AbortController | null = null;
|
||||||
|
private flushTimer: NodeJS.Timeout | null = null;
|
||||||
|
private stopped = false;
|
||||||
|
private backoff = 0;
|
||||||
|
private firstConnect = true;
|
||||||
|
|
||||||
|
constructor(options: SSEClientOptions) {
|
||||||
|
this.opts = {
|
||||||
|
url: options.url,
|
||||||
|
cursorPath: options.cursorPath,
|
||||||
|
onEvent: options.onEvent,
|
||||||
|
onGap: options.onGap,
|
||||||
|
logTag: options.logTag ?? '[SSE]',
|
||||||
|
reconnectBaseMs: options.reconnectBaseMs ?? 500,
|
||||||
|
reconnectMaxMs: options.reconnectMaxMs ?? 30_000,
|
||||||
|
flushIntervalMs: options.flushIntervalMs ?? 2000,
|
||||||
|
initialCursor: options.initialCursor,
|
||||||
|
};
|
||||||
|
this.backoff = this.opts.reconnectBaseMs;
|
||||||
|
}
|
||||||
|
|
||||||
|
async start(): Promise<void> {
|
||||||
|
await this.loadCursor();
|
||||||
|
if (!this.cursor && this.opts.initialCursor) {
|
||||||
|
this.cursor = this.opts.initialCursor;
|
||||||
|
}
|
||||||
|
this.startFlushTimer();
|
||||||
|
await this.runLoop();
|
||||||
|
}
|
||||||
|
|
||||||
|
async stop(): Promise<void> {
|
||||||
|
this.stopped = true;
|
||||||
|
if (this.abort) {
|
||||||
|
this.abort.abort();
|
||||||
|
this.abort = null;
|
||||||
|
}
|
||||||
|
if (this.flushTimer) {
|
||||||
|
clearInterval(this.flushTimer);
|
||||||
|
this.flushTimer = null;
|
||||||
|
}
|
||||||
|
await this.flushCursor();
|
||||||
|
}
|
||||||
|
|
||||||
|
getCursor(): string {
|
||||||
|
return this.cursor;
|
||||||
|
}
|
||||||
|
|
||||||
|
private async loadCursor(): Promise<void> {
|
||||||
|
if (!this.opts.cursorPath) return;
|
||||||
|
try {
|
||||||
|
const data = await fs.readFile(this.opts.cursorPath, 'utf-8');
|
||||||
|
const trimmed = data.trim();
|
||||||
|
if (trimmed) {
|
||||||
|
this.cursor = trimmed;
|
||||||
|
console.log(`${this.opts.logTag} Resuming from cursor ${this.cursor}`);
|
||||||
|
}
|
||||||
|
} catch (err: any) {
|
||||||
|
if (err.code !== 'ENOENT') {
|
||||||
|
console.warn(`${this.opts.logTag} Could not read cursor file ${this.opts.cursorPath}: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private startFlushTimer(): void {
|
||||||
|
this.flushTimer = setInterval(() => {
|
||||||
|
if (this.cursorDirty) {
|
||||||
|
this.flushCursor().catch(err =>
|
||||||
|
console.warn(`${this.opts.logTag} Cursor flush failed: ${err.message}`)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}, this.opts.flushIntervalMs);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async flushCursor(): Promise<void> {
|
||||||
|
if (!this.opts.cursorPath || !this.cursorDirty) return;
|
||||||
|
this.cursorDirty = false;
|
||||||
|
try {
|
||||||
|
await fs.mkdir(dirname(this.opts.cursorPath), { recursive: true });
|
||||||
|
await fs.writeFile(this.opts.cursorPath, this.cursor, 'utf-8');
|
||||||
|
} catch (err: any) {
|
||||||
|
this.cursorDirty = true;
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async runLoop(): Promise<void> {
|
||||||
|
while (!this.stopped) {
|
||||||
|
try {
|
||||||
|
await this.connect();
|
||||||
|
this.backoff = this.opts.reconnectBaseMs;
|
||||||
|
} catch (err: any) {
|
||||||
|
if (this.stopped) return;
|
||||||
|
console.warn(`${this.opts.logTag} Stream error, reconnecting in ${this.backoff}ms: ${err?.message ?? err}`);
|
||||||
|
await this.sleep(this.backoff);
|
||||||
|
this.backoff = Math.min(this.backoff * 2, this.opts.reconnectMaxMs);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async connect(): Promise<void> {
|
||||||
|
this.abort = new AbortController();
|
||||||
|
const headers: Record<string, string> = {
|
||||||
|
Accept: 'text/event-stream',
|
||||||
|
'Cache-Control': 'no-cache',
|
||||||
|
};
|
||||||
|
if (this.cursor) {
|
||||||
|
headers['Last-Event-ID'] = this.cursor;
|
||||||
|
}
|
||||||
|
this.firstConnect = false;
|
||||||
|
|
||||||
|
const response = await fetch(this.opts.url, {
|
||||||
|
method: 'GET',
|
||||||
|
headers,
|
||||||
|
signal: this.abort.signal,
|
||||||
|
});
|
||||||
|
if (!response.ok) throw new Error(`HTTP ${response.status} ${response.statusText}`);
|
||||||
|
if (!response.body) throw new Error('Response has no body');
|
||||||
|
|
||||||
|
const reader = response.body.getReader();
|
||||||
|
const decoder = new TextDecoder('utf-8');
|
||||||
|
let buffer = '';
|
||||||
|
|
||||||
|
try {
|
||||||
|
while (!this.stopped) {
|
||||||
|
const { value, done } = await reader.read();
|
||||||
|
if (done) return;
|
||||||
|
buffer += decoder.decode(value, { stream: true });
|
||||||
|
let idx: number;
|
||||||
|
while ((idx = buffer.indexOf('\n\n')) !== -1) {
|
||||||
|
const rawChunk = buffer.slice(0, idx);
|
||||||
|
buffer = buffer.slice(idx + 2);
|
||||||
|
await this.handleChunk(rawChunk);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
try { reader.releaseLock(); } catch { /* nothing useful */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async handleChunk(chunk: string): Promise<void> {
|
||||||
|
let id = '';
|
||||||
|
let event = '';
|
||||||
|
const dataLines: string[] = [];
|
||||||
|
for (const rawLine of chunk.split('\n')) {
|
||||||
|
if (!rawLine || rawLine.startsWith(':')) continue;
|
||||||
|
const colon = rawLine.indexOf(':');
|
||||||
|
if (colon === -1) continue;
|
||||||
|
const field = rawLine.slice(0, colon);
|
||||||
|
let value = rawLine.slice(colon + 1);
|
||||||
|
if (value.startsWith(' ')) value = value.slice(1);
|
||||||
|
switch (field) {
|
||||||
|
case 'id': id = value; break;
|
||||||
|
case 'event': event = value; break;
|
||||||
|
case 'data': dataLines.push(value); break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (dataLines.length === 0) return;
|
||||||
|
|
||||||
|
let data: any;
|
||||||
|
try {
|
||||||
|
data = JSON.parse(dataLines.join('\n'));
|
||||||
|
} catch (err: any) {
|
||||||
|
console.warn(`${this.opts.logTag} Skipping unparseable event ${id || '(no id)'}: ${err.message}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const ev: SSEEvent = { id, event: event || 'message', data };
|
||||||
|
if (ev.event === 'gap') {
|
||||||
|
if (this.opts.onGap) {
|
||||||
|
await this.opts.onGap(ev.data);
|
||||||
|
} else {
|
||||||
|
console.warn(`${this.opts.logTag} Gap event (cursor ${ev.data?.requested} trimmed): resumed from ${ev.data?.resumeFrom}`);
|
||||||
|
}
|
||||||
|
if (typeof ev.data?.resumeFrom === 'string' && ev.data.resumeFrom) {
|
||||||
|
this.cursor = ev.data.resumeFrom;
|
||||||
|
this.cursorDirty = true;
|
||||||
|
}
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
await this.opts.onEvent(ev);
|
||||||
|
} catch (err: any) {
|
||||||
|
console.error(`${this.opts.logTag} onEvent handler failed for ${ev.id}: ${err?.stack ?? err}`);
|
||||||
|
}
|
||||||
|
if (ev.id) {
|
||||||
|
this.cursor = ev.id;
|
||||||
|
this.cursorDirty = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private sleep(ms: number): Promise<void> {
|
||||||
|
return new Promise(resolve => setTimeout(resolve, ms));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -13,7 +13,6 @@
|
|||||||
*/
|
*/
|
||||||
export interface MediaMeta {
|
export interface MediaMeta {
|
||||||
// Title information
|
// Title information
|
||||||
titles?: { eng?: string; [key: string]: string | undefined };
|
|
||||||
originalTitle?: string;
|
originalTitle?: string;
|
||||||
title?: string;
|
title?: string;
|
||||||
|
|
||||||
@@ -62,7 +61,8 @@ export function renamingRule(metadata: Partial<MediaMeta>, filepath?: string): s
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Get title - try various fields
|
// Get title - try various fields
|
||||||
const title = metadata.titles?.eng || metadata.originalTitle;
|
// The display name — never the `titles/*` name set (METADATA_KEYS.md §3).
|
||||||
|
const title = metadata.title || metadata.originalTitle;
|
||||||
if (!title) {
|
if (!title) {
|
||||||
if (metadata.fileType !== 'torrent') {
|
if (metadata.fileType !== 'torrent') {
|
||||||
// Don't warn for torrent files because they don't always have a title
|
// Don't warn for torrent files because they don't always have a title
|
||||||
|
|||||||
@@ -174,12 +174,6 @@ export class VirtualFileSystem implements VFSUpdateCallback {
|
|||||||
mtime = isNaN(parsed) ? parseFloat(props.mtime) : parsed;
|
mtime = isNaN(parsed) ? parseFloat(props.mtime) : parsed;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Parse titles - can be from nested paths like 'titles/eng'
|
|
||||||
let titles: { eng?: string; [key: string]: string | undefined } | undefined;
|
|
||||||
if (props['titles/eng']) {
|
|
||||||
titles = { eng: props['titles/eng'] };
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse season/episode
|
// Parse season/episode
|
||||||
const season = props.season !== undefined ? parseInt(props.season, 10) : undefined;
|
const season = props.season !== undefined ? parseInt(props.season, 10) : undefined;
|
||||||
const episode = props.episode !== undefined ? parseInt(props.episode, 10) : undefined;
|
const episode = props.episode !== undefined ? parseInt(props.episode, 10) : undefined;
|
||||||
@@ -212,7 +206,6 @@ export class VirtualFileSystem implements VFSUpdateCallback {
|
|||||||
ctime: props.ctime ? parseFloat(props.ctime) : mtime,
|
ctime: props.ctime ? parseFloat(props.ctime) : mtime,
|
||||||
hashId,
|
hashId,
|
||||||
title: props.title,
|
title: props.title,
|
||||||
titles,
|
|
||||||
originalTitle: props.originalTitle,
|
originalTitle: props.originalTitle,
|
||||||
fileName,
|
fileName,
|
||||||
season,
|
season,
|
||||||
|
|||||||
@@ -7,7 +7,7 @@
|
|||||||
* Old behavior:
|
* Old behavior:
|
||||||
* - Only processes video, subtitle, torrent files
|
* - Only processes video, subtitle, torrent files
|
||||||
* - Requires extension
|
* - Requires extension
|
||||||
* - Title from titles.eng || originalTitle
|
* - Title from title || originalTitle
|
||||||
* - Files without title/extension are hidden (not in Unsorted)
|
* - Files without title/extension are hidden (not in Unsorted)
|
||||||
*/
|
*/
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ export type ConditionOperator =
|
|||||||
*/
|
*/
|
||||||
export interface Condition {
|
export interface Condition {
|
||||||
type: ConditionOperator;
|
type: ConditionOperator;
|
||||||
field: string; // Metadata field path, e.g., "titles.eng", "season", "fileType"
|
field: string; // Metadata field path, e.g., "title", "season", "fileType"
|
||||||
value?: string | number | boolean; // Required for EQUALS, NOT_EQUALS, CONTAINS, MATCHES
|
value?: string | number | boolean; // Required for EQUALS, NOT_EQUALS, CONTAINS, MATCHES
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -83,7 +83,7 @@ export interface PreviewResponse {
|
|||||||
*/
|
*/
|
||||||
export interface TemplateVariable {
|
export interface TemplateVariable {
|
||||||
name: string; // Display name
|
name: string; // Display name
|
||||||
path: string; // Field path (e.g., "titles.eng")
|
path: string; // Field path (e.g., "title")
|
||||||
type: 'string' | 'number' | 'boolean';
|
type: 'string' | 'number' | 'boolean';
|
||||||
description: string;
|
description: string;
|
||||||
example?: string;
|
example?: string;
|
||||||
@@ -93,7 +93,6 @@ export interface TemplateVariable {
|
|||||||
* List of all available template variables
|
* List of all available template variables
|
||||||
*/
|
*/
|
||||||
export const TEMPLATE_VARIABLES: TemplateVariable[] = [
|
export const TEMPLATE_VARIABLES: TemplateVariable[] = [
|
||||||
{ name: 'English Title', path: 'titles.eng', type: 'string', description: 'English title from metadata', example: 'Breaking Bad' },
|
|
||||||
{ name: 'Original Title', path: 'originalTitle', type: 'string', description: 'Original/native title', example: 'Breaking Bad' },
|
{ name: 'Original Title', path: 'originalTitle', type: 'string', description: 'Original/native title', example: 'Breaking Bad' },
|
||||||
{ name: 'Title', path: 'title', type: 'string', description: 'Simple title field', example: 'Breaking Bad' },
|
{ name: 'Title', path: 'title', type: 'string', description: 'Simple title field', example: 'Breaking Bad' },
|
||||||
{ name: 'Season', path: 'season', type: 'number', description: 'Season number', example: '1' },
|
{ name: 'Season', path: 'season', type: 'number', description: 'Season number', example: '1' },
|
||||||
|
|||||||
@@ -0,0 +1,138 @@
|
|||||||
|
/**
|
||||||
|
* TokenStore — Per-device WebDAV access tokens.
|
||||||
|
*
|
||||||
|
* Tokens are minted from the OIDC-protected dashboard and used as the
|
||||||
|
* basic-auth password by WebDAV clients (Plex, Finder, etc.). The plaintext
|
||||||
|
* is shown to the user once at creation time; only sha256 hashes are
|
||||||
|
* persisted. The matching WsgiDAV DomainController
|
||||||
|
* (`docker/webdav_token_controller.py`) reads the same JSON file fresh on
|
||||||
|
* every auth request, so revocation is immediate.
|
||||||
|
*
|
||||||
|
* Tokens identify a *device*, not an OIDC user — multi-user support requires
|
||||||
|
* hash-lock to forward an X-Forwarded-User header (not currently the case),
|
||||||
|
* at which point a `userId` field can be added to the record.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import * as crypto from 'crypto';
|
||||||
|
import * as fs from 'fs';
|
||||||
|
import * as path from 'path';
|
||||||
|
import { Logger } from 'tslog';
|
||||||
|
|
||||||
|
const logger = new Logger({ name: 'TokenStore' });
|
||||||
|
|
||||||
|
/** Length of the random portion (in hex chars) of a generated token. */
|
||||||
|
const TOKEN_RANDOM_HEX_LEN = 32; // 16 bytes => 128 bits of entropy
|
||||||
|
/** Public prefix marker so support can grep logs for stray tokens. */
|
||||||
|
const TOKEN_PREFIX = 'mfwd_';
|
||||||
|
/** Number of hex chars (after the prefix) we surface in the dashboard list. */
|
||||||
|
const VISIBLE_PREFIX_LEN = 4;
|
||||||
|
|
||||||
|
export interface WebdavTokenRecord {
|
||||||
|
id: string;
|
||||||
|
label: string;
|
||||||
|
hash: string; // sha256(plaintext)
|
||||||
|
prefix: string; // first VISIBLE_PREFIX_LEN chars of the random portion
|
||||||
|
createdAt: string; // ISO 8601
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Public view shown in the dashboard list — no plaintext, no hash. */
|
||||||
|
export interface WebdavTokenView {
|
||||||
|
id: string;
|
||||||
|
label: string;
|
||||||
|
prefix: string;
|
||||||
|
createdAt: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface TokenFile {
|
||||||
|
tokens: WebdavTokenRecord[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TokenStoreOptions {
|
||||||
|
configDir: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class TokenStore {
|
||||||
|
private readonly file: string;
|
||||||
|
private readonly configDir: string;
|
||||||
|
|
||||||
|
constructor(opts: TokenStoreOptions) {
|
||||||
|
this.configDir = opts.configDir;
|
||||||
|
this.file = path.join(this.configDir, 'webdav-tokens.json');
|
||||||
|
this.ensureConfigDir();
|
||||||
|
}
|
||||||
|
|
||||||
|
private ensureConfigDir(): void {
|
||||||
|
if (!fs.existsSync(this.configDir)) {
|
||||||
|
fs.mkdirSync(this.configDir, { recursive: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private read(): TokenFile {
|
||||||
|
if (!fs.existsSync(this.file)) return { tokens: [] };
|
||||||
|
try {
|
||||||
|
const raw = fs.readFileSync(this.file, 'utf-8');
|
||||||
|
const parsed = JSON.parse(raw) as TokenFile;
|
||||||
|
if (!Array.isArray(parsed.tokens)) return { tokens: [] };
|
||||||
|
return parsed;
|
||||||
|
} catch (err) {
|
||||||
|
logger.warn(`Token store ${this.file} unreadable, treating as empty:`, err);
|
||||||
|
return { tokens: [] };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Atomic write — write to a sibling tmp file and rename. WsgiDAV's
|
||||||
|
* controller reads this file on every auth request, so a partial write
|
||||||
|
* would briefly fail authentication.
|
||||||
|
*/
|
||||||
|
private write(file: TokenFile): void {
|
||||||
|
const tmp = `${this.file}.tmp`;
|
||||||
|
fs.writeFileSync(tmp, JSON.stringify(file, null, 2), 'utf-8');
|
||||||
|
fs.renameSync(tmp, this.file);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Public list — no plaintext, no hash. */
|
||||||
|
list(): WebdavTokenView[] {
|
||||||
|
return this.read().tokens.map(({ id, label, prefix, createdAt }) => ({
|
||||||
|
id, label, prefix, createdAt,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mint a new token. Returns the plaintext exactly once — the caller is
|
||||||
|
* responsible for surfacing it to the user and forgetting it.
|
||||||
|
*/
|
||||||
|
create(label: string): { plaintext: string; view: WebdavTokenView } {
|
||||||
|
const random = crypto.randomBytes(TOKEN_RANDOM_HEX_LEN / 2).toString('hex');
|
||||||
|
const plaintext = `${TOKEN_PREFIX}${random}`;
|
||||||
|
const id = crypto.randomBytes(8).toString('hex');
|
||||||
|
const record: WebdavTokenRecord = {
|
||||||
|
id,
|
||||||
|
label: label.trim() || 'Unnamed token',
|
||||||
|
hash: crypto.createHash('sha256').update(plaintext).digest('hex'),
|
||||||
|
prefix: random.slice(0, VISIBLE_PREFIX_LEN),
|
||||||
|
createdAt: new Date().toISOString(),
|
||||||
|
};
|
||||||
|
|
||||||
|
const file = this.read();
|
||||||
|
file.tokens.push(record);
|
||||||
|
this.write(file);
|
||||||
|
|
||||||
|
logger.info(`Minted WebDAV token id=${id} label=${JSON.stringify(record.label)}`);
|
||||||
|
return {
|
||||||
|
plaintext,
|
||||||
|
view: { id, label: record.label, prefix: record.prefix, createdAt: record.createdAt },
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Delete by id. Returns true if a record was removed. */
|
||||||
|
revoke(id: string): boolean {
|
||||||
|
const file = this.read();
|
||||||
|
const idx = file.tokens.findIndex(t => t.id === id);
|
||||||
|
if (idx < 0) return false;
|
||||||
|
file.tokens.splice(idx, 1);
|
||||||
|
this.write(file);
|
||||||
|
logger.info(`Revoked WebDAV token id=${id}`);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
import { useState, useEffect, useCallback } from 'react';
|
import { useState, useEffect, useCallback } from 'react';
|
||||||
import { RulesEditor } from './components/RulesEditor';
|
import { RulesEditor } from './components/RulesEditor';
|
||||||
import ServiceNav from './components/ServiceNav';
|
import ServiceNav from './components/ServiceNav';
|
||||||
|
import { WebdavTokens } from './components/WebdavTokens';
|
||||||
|
|
||||||
interface Stats {
|
interface Stats {
|
||||||
fileCount: number;
|
fileCount: number;
|
||||||
@@ -39,6 +40,7 @@ function App() {
|
|||||||
const [refreshing, setRefreshing] = useState(false);
|
const [refreshing, setRefreshing] = useState(false);
|
||||||
const [showRules, setShowRules] = useState(false);
|
const [showRules, setShowRules] = useState(false);
|
||||||
const [showSetup, setShowSetup] = useState(false);
|
const [showSetup, setShowSetup] = useState(false);
|
||||||
|
const [showTokens, setShowTokens] = useState(false);
|
||||||
|
|
||||||
const fetchData = useCallback(async () => {
|
const fetchData = useCallback(async () => {
|
||||||
try {
|
try {
|
||||||
@@ -223,6 +225,17 @@ function App() {
|
|||||||
<RulesEditor onSave={fetchData} />
|
<RulesEditor onSave={fetchData} />
|
||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
<div className="subsection">
|
||||||
|
<div
|
||||||
|
className="section-header-collapsible"
|
||||||
|
onClick={() => setShowTokens(!showTokens)}
|
||||||
|
>
|
||||||
|
<h3 className="subsection-title">WebDAV Access Tokens</h3>
|
||||||
|
<span className="collapse-icon">{showTokens ? '-' : '+'}</span>
|
||||||
|
</div>
|
||||||
|
{showTokens && <WebdavTokens />}
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="section">
|
<div className="section">
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ interface RulesEditorProps {
|
|||||||
// Available metadata fields for conditions
|
// Available metadata fields for conditions
|
||||||
const AVAILABLE_FIELDS = [
|
const AVAILABLE_FIELDS = [
|
||||||
{ value: 'fileType', label: 'File Type', type: 'string' },
|
{ value: 'fileType', label: 'File Type', type: 'string' },
|
||||||
{ value: 'titles.eng', label: 'English Title', type: 'string' },
|
{ value: 'title', label: 'Title', type: 'string' },
|
||||||
{ value: 'originalTitle', label: 'Original Title', type: 'string' },
|
{ value: 'originalTitle', label: 'Original Title', type: 'string' },
|
||||||
{ value: 'season', label: 'Season', type: 'number' },
|
{ value: 'season', label: 'Season', type: 'number' },
|
||||||
{ value: 'episode', label: 'Episode', type: 'number' },
|
{ value: 'episode', label: 'Episode', type: 'number' },
|
||||||
@@ -53,7 +53,7 @@ function createEmptyRule(): RenamingRule {
|
|||||||
function createEmptyCondition(): Condition {
|
function createEmptyCondition(): Condition {
|
||||||
return {
|
return {
|
||||||
type: 'EXISTS',
|
type: 'EXISTS',
|
||||||
field: 'titles.eng',
|
field: 'title',
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -197,7 +197,7 @@ function TemplateEditor({ template, onChange, variables }: TemplateEditorProps)
|
|||||||
<textarea
|
<textarea
|
||||||
value={template}
|
value={template}
|
||||||
onChange={e => onChange(e.target.value)}
|
onChange={e => onChange(e.target.value)}
|
||||||
placeholder="e.g., Movies/{titles.eng|originalTitle}{movieYear? ({movieYear})}/{fileName}.{extension}"
|
placeholder="e.g., Movies/{title|originalTitle}{movieYear? ({movieYear})}/{fileName}.{extension}"
|
||||||
rows={3}
|
rows={3}
|
||||||
className="template-input"
|
className="template-input"
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -0,0 +1,177 @@
|
|||||||
|
import { useCallback, useEffect, useState } from 'react';
|
||||||
|
|
||||||
|
interface TokenView {
|
||||||
|
id: string;
|
||||||
|
label: string;
|
||||||
|
prefix: string;
|
||||||
|
createdAt: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
interface CreatedToken extends TokenView {
|
||||||
|
token: string; // plaintext, surfaced only once
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatDate(iso: string): string {
|
||||||
|
try {
|
||||||
|
return new Date(iso).toLocaleString();
|
||||||
|
} catch {
|
||||||
|
return iso;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function WebdavTokens() {
|
||||||
|
const [tokens, setTokens] = useState<TokenView[]>([]);
|
||||||
|
const [loading, setLoading] = useState(true);
|
||||||
|
const [error, setError] = useState<string | null>(null);
|
||||||
|
const [newLabel, setNewLabel] = useState('');
|
||||||
|
const [creating, setCreating] = useState(false);
|
||||||
|
const [justCreated, setJustCreated] = useState<CreatedToken | null>(null);
|
||||||
|
const [copied, setCopied] = useState(false);
|
||||||
|
|
||||||
|
const refresh = useCallback(async () => {
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/webdav-tokens');
|
||||||
|
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||||
|
const data = await res.json();
|
||||||
|
setTokens(data.tokens ?? []);
|
||||||
|
setError(null);
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : String(err));
|
||||||
|
} finally {
|
||||||
|
setLoading(false);
|
||||||
|
}
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
refresh();
|
||||||
|
}, [refresh]);
|
||||||
|
|
||||||
|
const create = async () => {
|
||||||
|
setCreating(true);
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/webdav-tokens', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ label: newLabel }),
|
||||||
|
});
|
||||||
|
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||||
|
const created: CreatedToken = await res.json();
|
||||||
|
setJustCreated(created);
|
||||||
|
setNewLabel('');
|
||||||
|
setCopied(false);
|
||||||
|
await refresh();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : String(err));
|
||||||
|
} finally {
|
||||||
|
setCreating(false);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const revoke = async (id: string, label: string) => {
|
||||||
|
if (!confirm(`Revoke token "${label}"? Devices using it will lose WebDAV access immediately.`)) return;
|
||||||
|
try {
|
||||||
|
const res = await fetch(`/api/webdav-tokens/${id}`, { method: 'DELETE' });
|
||||||
|
if (!res.ok && res.status !== 204) throw new Error(`HTTP ${res.status}`);
|
||||||
|
await refresh();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : String(err));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const copyToken = async () => {
|
||||||
|
if (!justCreated) return;
|
||||||
|
try {
|
||||||
|
await navigator.clipboard.writeText(justCreated.token);
|
||||||
|
setCopied(true);
|
||||||
|
setTimeout(() => setCopied(false), 2000);
|
||||||
|
} catch {
|
||||||
|
// Clipboard API may be unavailable on http; fall back silently.
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if (loading) return <div className="loading">Loading tokens…</div>;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="webdav-tokens">
|
||||||
|
<p className="info-note">
|
||||||
|
Generate one token per device and paste it as the basic-auth password
|
||||||
|
when you add the WebDAV mount in Plex / Finder / Windows Explorer / etc.
|
||||||
|
Any username works — the password is what authenticates. Revoke a
|
||||||
|
token to cut off that single device without affecting the others.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
{error && <div className="error">{error}</div>}
|
||||||
|
|
||||||
|
{justCreated && (
|
||||||
|
<div className="info-box" style={{ borderColor: '#10b981' }}>
|
||||||
|
<h4>New token — copy it now</h4>
|
||||||
|
<p className="info-note">
|
||||||
|
This is the only time the full token will be shown. After you close
|
||||||
|
this banner only the prefix will appear in the list.
|
||||||
|
</p>
|
||||||
|
<div className="command-box" style={{ display: 'flex', gap: '0.5rem', alignItems: 'center' }}>
|
||||||
|
<code style={{ flex: 1, wordBreak: 'break-all' }}>{justCreated.token}</code>
|
||||||
|
<button onClick={copyToken} className="refresh-btn">
|
||||||
|
{copied ? 'Copied!' : 'Copy'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
onClick={() => setJustCreated(null)}
|
||||||
|
className="refresh-btn"
|
||||||
|
style={{ marginTop: '0.5rem' }}
|
||||||
|
>
|
||||||
|
Dismiss
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div className="card" style={{ marginTop: '1rem' }}>
|
||||||
|
<div className="card-header">
|
||||||
|
<span className="card-title">Generate token</span>
|
||||||
|
</div>
|
||||||
|
<div style={{ display: 'flex', gap: '0.5rem', marginTop: '0.5rem' }}>
|
||||||
|
<input
|
||||||
|
type="text"
|
||||||
|
value={newLabel}
|
||||||
|
onChange={(e) => setNewLabel(e.target.value)}
|
||||||
|
placeholder="Label (e.g. 'Plex on living-room TV')"
|
||||||
|
maxLength={200}
|
||||||
|
style={{ flex: 1, padding: '0.5rem' }}
|
||||||
|
onKeyDown={(e) => { if (e.key === 'Enter' && !creating) create(); }}
|
||||||
|
/>
|
||||||
|
<button onClick={create} disabled={creating} className="refresh-btn">
|
||||||
|
{creating ? 'Generating…' : 'Generate'}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="file-list" style={{ marginTop: '1rem' }}>
|
||||||
|
{tokens.length === 0 ? (
|
||||||
|
<div className="file-item">
|
||||||
|
<span className="file-name" style={{ color: '#64748b' }}>
|
||||||
|
No tokens yet — generate one to enable WebDAV access.
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
) : (
|
||||||
|
tokens.map((t) => (
|
||||||
|
<div key={t.id} className="file-item" style={{ justifyContent: 'space-between' }}>
|
||||||
|
<div>
|
||||||
|
<span className="file-name"><strong>{t.label}</strong></span>
|
||||||
|
<div className="info-note">
|
||||||
|
<code>mfwd_{t.prefix}…</code> · created {formatDate(t.createdAt)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<button
|
||||||
|
onClick={() => revoke(t.id, t.label)}
|
||||||
|
className="refresh-btn"
|
||||||
|
style={{ background: '#ef4444', color: 'white' }}
|
||||||
|
>
|
||||||
|
Revoke
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
))
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user