watch-skill

A skill (oxbshw, MIT) that gives any coding agent two abilities: watch video, and watch its own work and fix it. It turns a video — from a file, a screen recording, a live stream, or 1,800+ yt-dlp sources — into a searchable, persistent index of frames, on-screen text, and transcript, then exposes that to an agent over MCP (13 tools), a CLI, and a REST/OpenAPI surface. The second ability is the interesting one here: it packages output-grounded verification as a reusable, cross-harness skill rather than one repo’s bespoke loop.

The pipeline

Scene-aware frames — PySceneDetect + perceptual-hash dedup spends a frame budget (≤100/video) on distinct content, not fixed intervals.
OCR — RapidOCR, multi-script (Arabic, Cyrillic, Devanagari, Korean, …).
Transcription — offline by default via local faster-whisper; cloud STT optional.
Persistent index — SQLite + FTS5 full-text and ONNX/fastembed embeddings for hybrid retrieval, so a video answered once stays queryable.
Confidence scoring — calibrated answers with escalation ladders and verification passes. Built on yt-dlp, ffmpeg, PySceneDetect, RapidOCR, faster-whisper, fastembed. 284 offline tests; ~1.2s cold CLI start; a 10s clip processes in ~33s with no GPU.

THE LOOP — verification made perceptual

The self-verification mode: the agent records its own output (a screen recording of the UI it built, say), gets a structured critique against pass criteria, implements fixes, and re-verifies — then auto-generates a before/after proof GIF. This is the wiki’s loop + output-grounded-verification pattern applied to a new modality. Where ai-job-search compiles LaTeX and reads the rendered PDF to verify the artifact, watch-skill watches the rendered video — verifying temporal/visual output a text check can’t see. It’s the sharpest answer yet to the “a loop is only as good as its feedback signal” problem for visual work: the feedback signal becomes the agent literally seeing what it produced. And unlike ai-job-search’s in-repo loop, it’s shipped as a portable skill (Claude Code — machine-tested — plus Claude Desktop, Cursor, Codex CLI, Cline, Windsurf, Gemini CLI, VS Code Copilot, or any REST client).

Why it matters here

Verification as a first-class, reusable capability. loop-engineering argues the binding constraint has moved from generation to verification; watch-skill is that constraint answered as a product — a drop-in feedback signal for any agent’s loop, not code you rewrite per project.
Local-first, reaches-the-agent-over-MCP. Same delivery shape as knowledge-rag (RAG over MCP) and recall (memory over hooks): the store/index (here, of video) reaches the agent as a tool, offline by default. It extends that local-first corner from text/memory to perception.
Token discipline built in. Text-first answers, semantic caching, and per-question budgets (a self-reported ~86.6k tokens saved across 9 questions) — the tokenmining/headroom “don’t feed the model raw” thread applied to the token-heaviest modality, video.

Cross-spoke

The dominant substance is an agent skill/verification capability, so it ingests here. It embeds STT (faster-whisper) and OCR as components, a faint adjacency to speech-audio-wiki (the STT/ASR model market) — but the subject is the agent capability, not the transcription model, so it’s cross-linked context, not a route there.

Tier

T1 — first-party MIT repo, primary on itself. Young (~113★ at snapshot) and the token-savings / timing / test-count figures are self-reported on the author’s dev machine, not independently reproduced (weakness recorded).

text-first-agent-design · video-use · loop-engineering · ai-job-search · agent-loops-verification · self-improving-agents · knowledge-rag · recall · model-context-protocol · agent-skills · tokenmining · synthesis