Intelligence Brief 06 — Technical Infrastructure

The engine.
For Peter’s team.

A Docker Compose stack on a dedicated VPS. Submit a YouTube URL — the pipeline transcribes, extracts, captions, and surfaces everything in a review dashboard. Human approval before anything goes live.

Docker
Compose stack
Python
3.12 backend
pgvector
Knowledge DB
Tailscale
Private access
docker-compose.yml — Stack Services
services: db: pgvector/pgvector:pg16 # PostgreSQL + vector search redis: redis:7-alpine # Job queue broker api: ./api # FastAPI — 0.0.0.0:8000 worker: ./api (celery) # Background agent worker dashboard: ./dashboard # Next.js — 0.0.0.0:3000 volumes: pg_data · redis_data · media_store (/media)
Component Stack
Orchestration
Docker Compose
Single command brings up the full stack — API, worker, dashboard, DB, Redis
Backend API
FastAPI (Python 3.12)
REST API for all pipeline operations — ingest, review, approve, publish
Job Queue
Celery + Redis
Long-running AI jobs run in background — transcription can take 10–20 min
Database
PostgreSQL 16 + pgvector
Stores all jobs, transcripts, clips, captions. pgvector for Knowledge DB semantic search
Downloader
yt-dlp
Pulls video + metadata from YouTube in best available quality (≤720p)
Transcription
OpenAI Whisper (local)
Runs on-VPS — no data leaves the machine. Timestamped transcript with segments
AI Intelligence
Anthropic Claude API
Selects best clip moments, writes platform captions, analyses performance
Dashboard
Next.js 15
Human review UI — approve/reject clips, edit captions, monitor pipeline status
VPS Access
Tailscale
Zero-config private network. Chris and Peter access the dashboard securely. Public demo URL via Funnel
Video Edit
DaVinci / Premiere API
Phase 3 — AI triggers rough cut assembly via scripting APIs
Pipeline — URL In, Captions Out
Submit YouTube URL
Dashboard → POST /ingest
Download video + metadata
yt-dlp → mp4 + JSON
Generate transcript
OpenAI Whisper (local, on-VPS)
Identify best clips
Claude API → JSON with timestamps
Cut clips
ffmpeg → clip_1.mp4, clip_2.mp4...
Write captions
Claude API → Instagram, YouTube, TikTok, X
Human review
Dashboard → approve / edit / reject
Load into scheduler
Phase 2 — social publishing queue
Known Technical Challenges
!
Grading of visual — colour consistency
Drone, camera and archive footage have different colour profiles. Phase 3 uses DaVinci colour node scripting to normalise; requires approved LUT from Jim’s editor.
!
Sound sync between different shots
Multi-source audio (lavalier, boom, room) requires alignment. ffmpeg auto-sync works for single-cam; multi-cam sync needs DaVinci scripting in Phase 3.
Access Model
VPS — DigitalOcean LON1
Spec
4 vCPU / 8 GB RAM + 100 GB volume
OS
Ubuntu 22.04 LTS
Access
Tailscale — private network only
Dashboard
http://jhwwii-lab:3000 (Tailscale hostname)
API docs
http://jhwwii-lab:8000/docs
Public demo
tailscale funnel --bg 3000 → HTTPS URL
CI access
Chris + Peter added to Tailscale network