3 Commits
Author SHA1 Message Date
Worph d35a64a744 align with metadata standard
Build and Push Docker Image / build (push) Failing after 15s
2026-09-13 15:57:12 +02:00
Worph 9061c7a37d gitattributes 2026-07-14 16:08:49 +02:00
Pierre b100b3a77c improve doc + bugfix 2026-05-25 23:47:48 +02:00
10 changed files with 43 additions and 45 deletions
+6
View File
@@ -0,0 +1,6 @@
# FIX CRLF always when developer has not set
# Linux/Mac: git config --global core.autocrlf input
# Windows: git config --global core.autocrlf true
# Auto detect text files and perform LF normalization
* text=auto
* eol=lf
+11 -11
View File
@@ -84,7 +84,7 @@ Meta-Fuse is a read-only process in the MetaMesh ecosystem that:
│ [3] meta-stremio (PROCESS-READ) │ │ [3] meta-stremio (PROCESS-READ) │
│ └─► Reads metadata from KV, streams media content │ │ └─► Reads metadata from KV, streams media content │
│ │ │ │
│ [4] meta-orbit (SHARING-READ-WRITE) │ │ [4] meta-share (SHARING-READ-WRITE) │
│ └─► P2P metadata sync across network │ │ └─► P2P metadata sync across network │
│ │ │ │
└────────────────────────────────────────────────────────────────────────────┘ └────────────────────────────────────────────────────────────────────────────┘
@@ -104,7 +104,7 @@ http://meta-core:9000
/urls API Response (JSON): /urls API Response (JSON):
{ {
"hostname": "meta-core-dev", "hostname": "metacore-app",
"baseUrl": "http://localhost:8083", "baseUrl": "http://localhost:8083",
"apiUrl": "http://meta-core:9000", "apiUrl": "http://meta-core:9000",
"redisUrl": "redis://meta-core:6379", "redisUrl": "redis://meta-core:6379",
@@ -606,10 +606,10 @@ journalctl -u meta-fuse-driver -f
cat /meta-core/locks/kv-leader.info cat /meta-core/locks/kv-leader.info
# Check /urls API # Check /urls API
curl -k https://dev.localhost:8083/api/urls curl -k https://metacore-dev.localhost:8083/api/urls
# Verify redis is running (on leader) # Verify redis is running (on leader)
docker exec meta-core-dev redis-cli ping docker exec metacore-app redis-cli ping
# Check meta-fuse logs # Check meta-fuse logs
docker logs meta-fuse | grep "KV" docker logs meta-fuse | grep "KV"
@@ -632,10 +632,10 @@ docker logs meta-fuse | grep "wsgidav"
```bash ```bash
# Check Redis has file data (flat key format) # Check Redis has file data (flat key format)
docker exec meta-core-dev redis-cli keys "file:*" | head -20 docker exec metacore-app redis-cli keys "file:*" | head -20
# Check file index # Check file index
docker exec meta-core-dev redis-cli scard "file:__index__" docker exec metacore-app redis-cli scard "file:__index__"
# Verify source files exist # Verify source files exist
ls -la /files/watch/ ls -la /files/watch/
@@ -644,7 +644,7 @@ ls -la /files/watch/
curl http://localhost:3000/api/fuse/stats curl http://localhost:3000/api/fuse/stats
# Check streaming state builder stats # Check streaming state builder stats
docker logs meta-fuse-dev | grep "State builder" docker logs metafuse-app | grep "State builder"
``` ```
## Technology Stack ## Technology Stack
@@ -665,7 +665,7 @@ Meta-Fuse is designed to work seamlessly with other MetaMesh services:
- **meta-sort**: Provides metadata in KV database - **meta-sort**: Provides metadata in KV database
- **meta-stremio**: Can stream files exposed by meta-fuse - **meta-stremio**: Can stream files exposed by meta-fuse
- **meta-orbit**: Syncs metadata across P2P network - **meta-share**: Syncs metadata across P2P network
### Connecting to meta-sort ### Connecting to meta-sort
@@ -673,13 +673,13 @@ Ensure meta-sort is running and has processed files:
```bash ```bash
# Check meta-sort processing status # Check meta-sort processing status
curl -k https://dev.localhost:8180/api/processing/status curl -k https://metasort-dev.localhost:8180/api/processing/status
# Verify Redis has metadata (via meta-core) # Verify Redis has metadata (via meta-core)
docker exec meta-core-dev redis-cli scard "file:__index__" docker exec metacore-app redis-cli scard "file:__index__"
# Check meta:events stream has events # Check meta:events stream has events
docker exec meta-core-dev redis-cli xlen "meta:events" docker exec metacore-app redis-cli xlen "meta:events"
``` ```
Meta-fuse will automatically discover the KV database through the shared lock file at `/meta-core/locks/kv-leader.info`. Meta-fuse will automatically discover the KV database through the shared lock file at `/meta-core/locks/kv-leader.info`.
+7 -7
View File
@@ -535,29 +535,29 @@ List all discovered MetaMesh services.
"services": [ "services": [
{ {
"name": "meta-sort", "name": "meta-sort",
"url": "http://meta-sort-dev", "url": "http://metasort-app",
"api": "http://meta-sort-dev/api", "api": "http://metasort-app/api",
"status": "healthy", "status": "healthy",
"role": "writer" "role": "writer"
}, },
{ {
"name": "meta-fuse", "name": "meta-fuse",
"url": "http://meta-fuse-dev", "url": "http://metafuse-app",
"api": "http://meta-fuse-dev/api", "api": "http://metafuse-app/api",
"status": "healthy", "status": "healthy",
"role": "reader" "role": "reader"
}, },
{ {
"name": "meta-stremio", "name": "meta-stremio",
"url": "http://meta-stremio-dev", "url": "http://metastremio-app",
"api": "http://meta-stremio-dev/api", "api": "http://metastremio-app/api",
"status": "healthy", "status": "healthy",
"role": "reader" "role": "reader"
} }
], ],
"current": "meta-fuse", "current": "meta-fuse",
"leader": { "leader": {
"hostname": "meta-core-dev", "hostname": "metacore-app",
"baseUrl": "http://localhost:8083", "baseUrl": "http://localhost:8083",
"apiUrl": "http://meta-core:9000", "apiUrl": "http://meta-core:9000",
"redisUrl": "redis://meta-core:6379", "redisUrl": "redis://meta-core:6379",
+1 -1
View File
@@ -350,7 +350,7 @@ The `onFileComplete` callback is called when:
``` ```
[meta-fuse] Starting meta-fuse... [meta-fuse] Starting meta-fuse...
[KVManager] Waiting for leader... [KVManager] Waiting for leader...
[LeaderClient] Leader found: meta-core-dev at redis://meta-core:6379 [LeaderClient] Leader found: metacore-app at redis://meta-core:6379
[meta-fuse] KV Manager ready, initializing VFS with streaming bootstrap... [meta-fuse] KV Manager ready, initializing VFS with streaming bootstrap...
[StreamingStateBuilder] Initialized with 12 VFS-relevant properties [StreamingStateBuilder] Initialized with 12 VFS-relevant properties
[meta-fuse] Replaying meta:events stream from position 0... [meta-fuse] Replaying meta:events stream from position 0...
+10 -10
View File
@@ -75,7 +75,6 @@ export interface FileMetadata {
// Title information (for folder organization) // Title information (for folder organization)
title?: string; title?: string;
titles?: { eng?: string; [key: string]: string | undefined };
originalTitle?: string; originalTitle?: string;
fileName?: string; fileName?: string;
@@ -623,12 +622,6 @@ export class RedisClient implements Partial<IKVClient> {
mtime = isNaN(parsed) ? parseFloat(data.mtime) : parsed; mtime = isNaN(parsed) ? parseFloat(data.mtime) : parsed;
} }
// Parse titles - can be nested object or simple string
let titles: { eng?: string; [key: string]: string | undefined } | undefined;
if (data['titles/eng']) {
titles = { eng: data['titles/eng'] };
}
// Parse season/episode (can be 0 for specials) // Parse season/episode (can be 0 for specials)
const season = data.season !== undefined ? parseInt(data.season, 10) : undefined; const season = data.season !== undefined ? parseInt(data.season, 10) : undefined;
const episode = data.episode !== undefined ? parseInt(data.episode, 10) : undefined; const episode = data.episode !== undefined ? parseInt(data.episode, 10) : undefined;
@@ -647,12 +640,19 @@ export class RedisClient implements Partial<IKVClient> {
mtime, mtime,
ctime: data.ctime ? parseFloat(data.ctime) : mtime, ctime: data.ctime ? parseFloat(data.ctime) : mtime,
// Identification // Identification.
hashId: hashId ?? data.cid_midhash256 ?? data.hashId, //
// `hashId` is always supplied by the caller — every `parseMetadata`
// call site derives it from the Redis key (`file:<hashId>/…`), so the
// fallbacks below it were already unreachable. The two that used to sit
// here (`data.cid_midhash256`, then `data.hashId`) were also the wrong
// shape: `cid_*` fields no longer exist on records, and meta-core
// rejects them at the write boundary. Kept as a single defensive `??`
// rather than a chain of dead ones. See METADATA_KEYS.md §14.13.
hashId: hashId ?? data.hashId,
// Title information // Title information
title: data.title, title: data.title,
titles,
originalTitle: data.originalTitle, originalTitle: data.originalTitle,
fileName: data.fileName, fileName: data.fileName,
@@ -13,7 +13,6 @@
*/ */
export interface MediaMeta { export interface MediaMeta {
// Title information // Title information
titles?: { eng?: string; [key: string]: string | undefined };
originalTitle?: string; originalTitle?: string;
title?: string; title?: string;
@@ -62,7 +61,8 @@ export function renamingRule(metadata: Partial<MediaMeta>, filepath?: string): s
} }
// Get title - try various fields // Get title - try various fields
const title = metadata.titles?.eng || metadata.originalTitle; // The display name — never the `titles/*` name set (METADATA_KEYS.md §3).
const title = metadata.title || metadata.originalTitle;
if (!title) { if (!title) {
if (metadata.fileType !== 'torrent') { if (metadata.fileType !== 'torrent') {
// Don't warn for torrent files because they don't always have a title // Don't warn for torrent files because they don't always have a title
@@ -174,12 +174,6 @@ export class VirtualFileSystem implements VFSUpdateCallback {
mtime = isNaN(parsed) ? parseFloat(props.mtime) : parsed; mtime = isNaN(parsed) ? parseFloat(props.mtime) : parsed;
} }
// Parse titles - can be from nested paths like 'titles/eng'
let titles: { eng?: string; [key: string]: string | undefined } | undefined;
if (props['titles/eng']) {
titles = { eng: props['titles/eng'] };
}
// Parse season/episode // Parse season/episode
const season = props.season !== undefined ? parseInt(props.season, 10) : undefined; const season = props.season !== undefined ? parseInt(props.season, 10) : undefined;
const episode = props.episode !== undefined ? parseInt(props.episode, 10) : undefined; const episode = props.episode !== undefined ? parseInt(props.episode, 10) : undefined;
@@ -212,7 +206,6 @@ export class VirtualFileSystem implements VFSUpdateCallback {
ctime: props.ctime ? parseFloat(props.ctime) : mtime, ctime: props.ctime ? parseFloat(props.ctime) : mtime,
hashId, hashId,
title: props.title, title: props.title,
titles,
originalTitle: props.originalTitle, originalTitle: props.originalTitle,
fileName, fileName,
season, season,
@@ -7,7 +7,7 @@
* Old behavior: * Old behavior:
* - Only processes video, subtitle, torrent files * - Only processes video, subtitle, torrent files
* - Requires extension * - Requires extension
* - Title from titles.eng || originalTitle * - Title from title || originalTitle
* - Files without title/extension are hidden (not in Unsorted) * - Files without title/extension are hidden (not in Unsorted)
*/ */
@@ -21,7 +21,7 @@ export type ConditionOperator =
*/ */
export interface Condition { export interface Condition {
type: ConditionOperator; type: ConditionOperator;
field: string; // Metadata field path, e.g., "titles.eng", "season", "fileType" field: string; // Metadata field path, e.g., "title", "season", "fileType"
value?: string | number | boolean; // Required for EQUALS, NOT_EQUALS, CONTAINS, MATCHES value?: string | number | boolean; // Required for EQUALS, NOT_EQUALS, CONTAINS, MATCHES
} }
@@ -83,7 +83,7 @@ export interface PreviewResponse {
*/ */
export interface TemplateVariable { export interface TemplateVariable {
name: string; // Display name name: string; // Display name
path: string; // Field path (e.g., "titles.eng") path: string; // Field path (e.g., "title")
type: 'string' | 'number' | 'boolean'; type: 'string' | 'number' | 'boolean';
description: string; description: string;
example?: string; example?: string;
@@ -93,7 +93,6 @@ export interface TemplateVariable {
* List of all available template variables * List of all available template variables
*/ */
export const TEMPLATE_VARIABLES: TemplateVariable[] = [ export const TEMPLATE_VARIABLES: TemplateVariable[] = [
{ name: 'English Title', path: 'titles.eng', type: 'string', description: 'English title from metadata', example: 'Breaking Bad' },
{ name: 'Original Title', path: 'originalTitle', type: 'string', description: 'Original/native title', example: 'Breaking Bad' }, { name: 'Original Title', path: 'originalTitle', type: 'string', description: 'Original/native title', example: 'Breaking Bad' },
{ name: 'Title', path: 'title', type: 'string', description: 'Simple title field', example: 'Breaking Bad' }, { name: 'Title', path: 'title', type: 'string', description: 'Simple title field', example: 'Breaking Bad' },
{ name: 'Season', path: 'season', type: 'number', description: 'Season number', example: '1' }, { name: 'Season', path: 'season', type: 'number', description: 'Season number', example: '1' },
@@ -16,7 +16,7 @@ interface RulesEditorProps {
// Available metadata fields for conditions // Available metadata fields for conditions
const AVAILABLE_FIELDS = [ const AVAILABLE_FIELDS = [
{ value: 'fileType', label: 'File Type', type: 'string' }, { value: 'fileType', label: 'File Type', type: 'string' },
{ value: 'titles.eng', label: 'English Title', type: 'string' }, { value: 'title', label: 'Title', type: 'string' },
{ value: 'originalTitle', label: 'Original Title', type: 'string' }, { value: 'originalTitle', label: 'Original Title', type: 'string' },
{ value: 'season', label: 'Season', type: 'number' }, { value: 'season', label: 'Season', type: 'number' },
{ value: 'episode', label: 'Episode', type: 'number' }, { value: 'episode', label: 'Episode', type: 'number' },
@@ -53,7 +53,7 @@ function createEmptyRule(): RenamingRule {
function createEmptyCondition(): Condition { function createEmptyCondition(): Condition {
return { return {
type: 'EXISTS', type: 'EXISTS',
field: 'titles.eng', field: 'title',
}; };
} }
@@ -197,7 +197,7 @@ function TemplateEditor({ template, onChange, variables }: TemplateEditorProps)
<textarea <textarea
value={template} value={template}
onChange={e => onChange(e.target.value)} onChange={e => onChange(e.target.value)}
placeholder="e.g., Movies/{titles.eng|originalTitle}{movieYear? ({movieYear})}/{fileName}.{extension}" placeholder="e.g., Movies/{title|originalTitle}{movieYear? ({movieYear})}/{fileName}.{extension}"
rows={3} rows={3}
className="template-input" className="template-input"
/> />