How much of a multi-camera shoot can an automatic edit recover, and what stops it?

What it found

A synthetic model of four cameras whose true angle quality changes every 2–8 s, seen through a noisy detector. At noise 0.3 a fixed 3 s rotation shows mean true quality 0.546; an offline Viterbi edit that maximises detected quality minus a cost per cut reaches 0.802 against 0.815 for the same search on true scores, while picking the best score every frame gives 0.723 with 345 cuts a minute and every shot under 1.5 s. The cut cost matters more than detector accuracy (best near 1.0, about 14 cuts a minute). Clock sync from a shared loudness envelope fails first on short clips: a 5 s clip at noise 0.25 is off by over 40 ms in 25% of trials, 15 s in 5%. No real footage, reframing or film-look grade; quality is a made-up random process.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Autonomous multi-angle capture

• When does automatic blurring miss a bystander's face, and which filming platform is conspicuous?When does automatic blurring miss a bystander's face? On an assumed size-recall model a face detector leaves 5% of identifiable faces unblurred per frame, blurring the head of any detected body under 0.1%, and independent per-frame misses show a face in some frame of a 5 s shot 100% of the time; an offline whole-track blur leaves 0 to 5%. No real footage or detector.
• Does recording only when something happens save a travel creator's battery, and what does it cost in missed moments?Does recording only on a trigger save battery, and what does it miss? With 6 thirty-second moments an hour, standby with a 10 s pre-roll uses 19.4 Wh of an 80 Wh continuous day (76% less) and captures 90% of interesting seconds; deep sleep uses 7.7 Wh but captures 69%. Misses and wake latency, not the battery, set coverage. Assumed device figures, synthetic day.
• When a camera rig records only on a trigger, should every camera have its own, or should one wake the rest?When a camera rig records only on a trigger, should every camera have its own, or should one wake the rest? Four standby cameras with their own triggers use 80 Wh of 320 Wh continuous but get all four angles on only 68% of interesting seconds; one sentinel waking the others costs the same and gives 90.5%, and any-camera wake gives 100% for 92 Wh but its false-alarm energy grows with detector count. Assumed device figures.
• Where in a camera ring should the sentinel go when the subject faces a favourite direction?Where in a camera ring should the sentinel go when the subject faces a favourite direction? A sentinel facing the favourite direction misses 8.6% of moments against 34.4% for the one behind; spread cameras' misses are anti-correlated, so capture-wake's independence assumption overstates the all-miss rate by 2x on a ring of 4 and understates it 12x for a tight cluster. With the turned-face miss measured in capture-detectors (54 faces) in place of the assumed curve, placement matters only if the back of a head goes undetected (untested), and weak detectors miss 8 to 45% of moments even with four cameras listening.
• Does a longer window, a lower gamma or a delay prior fix the period aliasing and early-start failures of GCC audio sync on real clips?Does a longer window or a delay prior fix GCC sync's aliasing on real clips? Window length (shared audio) is the lever: 97% wrong at 1 s, 38% at 2 s, 16% at 3 s on the failing clips; a tone clip needs a prior tighter than its period.
• How long a window of sound does it take to sync two drifting cameras, and how often must you redo it?How long a window of sound syncs two drifting cameras? GCC-PHAT plus a robust line fit syncs to under 0.02 ms over an hour from -10 dB SNR with 2 s windows; a sync made once at the start drifts one frame in 833 s at 40 ppm.

All 34 Autonomous multi-angle capture projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.