Karate 3D Avatar Studio

Turn any video into an interactive 3D avatar: detection, camera estimation and SAM 3D Body mesh recovery, with a Three.js studio for labelling techniques and voice control.

February 2026ShippedSource on GitHub ↗

A pipeline from video (local file or YouTube) to rigged, labelled 3D animation. Frames are extracted with OpenCV, people found with YOLOv8 or ViTDet, camera intrinsics estimated with MoGe2, and Meta's SAM 3D Body recovers a full mesh per frame, with optional SAM2 masks.

The output is exported as binary mesh data for a Three.js studio with a synced source video, timeline, and a segment editor for labelling stances, kicks and blocks. Voice commands such as "show me a horse stance" are resolved by Llama 3.3 70B to the matching segment.

Hard parts

Chaining five vision models into one pipeline
Streaming per-frame meshes to the browser as compact binary

More work

Prep: voice coding copilotAn interview-prep IDE with a voice: a Monaco editor, a LeetCode-style problem library, and an AI copilot you can talk to, interrupt and point at your code.
Agent2DBA desktop database agent: connect PostgreSQL, ask in plain language, and let an agent explore the schema, write and run SQL, and remember what it learns, with a human approving every write.
voiceAgentA fully local, real-time voice assistant you can talk over: Parakeet listens, an LLM thinks, Kokoro speaks, SQLite remembers. Sub-second speech-to-speech with two-stage barge-in.