Footage first: why we let humans mark the good parts
AI can describe a shot. It's surprisingly bad at telling whether handheld footage is too shaky to post.
Phone farm videos live or die on their footage. A calm shot of a rack of phones looks impressive; the same shot with a shaky hand looks cheap. So we tried to let AI decide which parts of a recording are usable. It didn't work well enough.
What went wrong
The AI described scenes accurately — "rack of phones", "dashboard with numbers" — but judged far too much handheld footage as steady. Videos built on its judgement looked rushed and shaky.
Measuring instead of guessing
We started measuring camera motion directly: every frame is compared with the previous one to see how far the picture moved, and how much that movement jitters. That gave objective numbers for shake and speed, calibrated on reference videos that performed well.
Even measured, a threshold can't capture taste. Some slow pans are fine; some tiny jolts are not.
The final design: suggest, then decide
- Recordings open with a suggested split — calm parts green, shaky parts red.
- Every part comes with reasons: measured shake and speed, plus the AI's notes on sharpness and interest.
- You confirm with a few keystrokes — G for good, B for bad — or drag across a stretch.
- Your marks teach the suggestions: the motion limits adapt to what you keep and what you throw out.
- Only confirmed parts are ever used, and each only in a few videos.
The lesson generalizes beyond footage: let AI do the tedious work of proposing, measuring and explaining — and keep the final call with the person who knows what good looks like.