Media Pipeline
backend/media-pipeline/ (Express + FFmpeg, HTTP port 3002, WebSocket port 3003)
handles cloud-side frame extraction and transcription for media too heavy to process in-browser.
Endpoints
| Method | Path | Purpose |
|---|---|---|
POST | /jobs | Submit a processing job (extraction, transcription) |
GET | /jobs/:id | Poll job status |
GET | /jobs/:id/frames | Retrieve extracted frames |
GET | /health | Liveness probe |
WS | :3003 | Live job progress streaming |
Job flow
- The client (
src/lib/media-pipeline.ts,useMediaPipeline) submits a job - FFmpeg extracts frames / audio server-side into the
media-tempvolume - Progress streams over the WebSocket — the UI shows live extraction progress
- Frames feed the Visionary and CLIP embeddings
Browser-local fallback
Without this service, frame extraction runs in the
WebWorker pool —
n = navigator.hardwareConcurrency workers with work-stealing — which is plenty for typical
clips and keeps everything on-device.