Computer vision and editing UX
Keep the right face in frame. Keep the editor in control.
A shot-aware system for stable portrait reframing, with precise corrections when automation needs help.
Automation gets most frames right. Editors handle the meaningful exceptions.
A face box is not a camera decision. Dialogue scenes introduce identity switches, occlusion, partial faces, multiple speakers, rapid cuts, and letterboxed footage. A crop can be mathematically valid and still feel visually wrong.
I designed the system around two paths: a strong automatic first pass for speed, and an intentional correction workspace for trust. Editors fix who the camera follows or where the crop sits. They do not rebuild the motion frame by frame.
From detection signal to portrait camera
Detect
RetinaFace detects faces while MediaPipe and YOLO-face remain available as runtime fallbacks.
Track
Spatial overlap and 512-D appearance embeddings stabilize identity across movement and temporary misses.
Choose
Talk, focus, face size, and shot boundaries help select the subject that should lead the frame.
Compose
The camera path is smoothed, constrained to source bounds, and adjusted for cinematic black bars.
Make the model legible
The QA render draws face IDs, boxes, talk scores, and focus scores on the original video. Tracking failures become visible evidence instead of a vague bad crop downstream.
The correction workspace edits intent
Follow a face
Click any tracked face to keep that identity as the subject for the current shot.
Place the composition
Click the portrait preview to choose where the selected face should sit, or drag the crop freely.
Review safely
Scrub frame by frame, adjust crop size, undo, reset the shot, and save a separate corrections document.
A complete, reusable review loop
One command produces normalized media, detection and tracking metadata, shot boundaries, cinematic-scope data, a visual QA render, and a prepared Remotion preview. Manual edits stay separate from face metadata and blend into the automatic camera path with smooth easing.
Python · OpenCV · InsightFace · FFmpeg · Remotion · React