How often must free-running cameras be re-synced by audio, and does rejecting wild estimates help?

What it found

Schedule study for audio sync of two free-running cameras on synthetic speech-like audio (capture-audiosync's GCC-PHAT estimator). With a 40 ms tolerance (EBU R37 lead limit, a stand-in for a cut threshold) an offset-only sync lasts 1000 s at 40 ppm, so even perfect estimates every 1800 s leave 44% of a 6 h shoot out of tolerance. A drift gate that refuses estimates a real clock (≤100 ppm) could not produce cuts that share from 35% to 22% at −12 dB SNR and from 11% to 3% at −10 dB when re-syncing every 300 s, and removes all wrongly applied syncs at −12 dB; at 900 s or longer it does nothing, since one skipped window already exceeds the tolerance. A wrong first sync is not fixable by a gate. Not real recordings: audio is synthetic, drift is a straight line at an assumed 40 ppm, and the tolerance was set for lip-sync, not for cuts between cameras.

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Autonomous multi-angle capture

• Does a residual audio-sync error show at the cuts of an automatic multi-camera edit?Does a residual audio-sync error show at the cuts of an automatic edit? A sync made once leaves 50% of cuts jumping over a frame; a robust fit leaves none down to -10 dB SNR (2 s windows) and then fails outright, with 11 to 24% of cuts repeating or dropping a loud event.
• Does a robust fit or a Kalman filter hold a camera's clock better between audio re-syncs than a least-squares line, once the clock drift comes from a real crystal curve?Does a robust fit or a Kalman filter hold a camera clock better between audio re-syncs? On a clock rate computed from a sourced AT-cut crystal curve the plain offset hold is already fine up to 1800 s and every fit ties; only a Kalman filter tuned to an assumed 30 ppm wobble wins. Synthetic audio.
• Does finding the start offset first let short audio windows sync two drifting cameras at lower SNR?Does a coarse offset first let short windows sync drifting cameras? A 10 s window finds the offset, then 0.5 s windows hold within a frame from -12 dB SNR; plain 1 s windows fail to -5 dB. Synthetic audio.
• Does dropping doubtful sync windows, with settings chosen on other clips, keep a multi-camera edit in sync on real audio?Does a peak-sharpness gate plus settings chosen on other clips keep the edit in sync? At 0 dB yes (1.9% of cuts over a frame held-out vs 34% for PHAT); at -10 dB only partly (39% vs 64%).
• Does holding a fitted clock slope between audio re-syncs let free-running cameras go longer without a refresh?Does holding a fitted clock slope between audio re-syncs help? At 40 ppm it cuts out-of-tolerance time from 44% to 3.7% at 1800 s intervals, but an assumed rate wobble makes a strict line gate lock out. Synthetic audio; also corrects capture-syncbudget.
• Can a delay prior taken from a camera's own earlier windows give the oracle-prior gain in GCC audio sync?Can a delay prior from a camera's own earlier windows replace the oracle? Not yet: 22.5% to 37% wrong on the failing clips against 0% for the oracle, and an early alias becomes an hour-long lock-in.

All 34 Autonomous multi-angle capture projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.