Does dropping doubtful sync windows, with settings chosen on other clips, keep a multi-camera edit in sync on real audio?
research/capture-syncgate
Does a peak-sharpness gate plus settings chosen on other clips keep the edit in sync? At 0 dB yes (1.9% of cuts over a frame held-out vs 34% for PHAT); at -10 dB only partly (39% vs 64%).
What it found
Follow-up to capture-realedit. Adds a peak-to-sidelobe (PSR) gate that drops GCC windows whose peak is not distinct before the Theil-Sen fit, and picks gamma and the gate threshold leave-one-clip-out over 10 real LibreOffice gallery clips (4 cameras, 12 windows, ±40 ppm, 12 seeds per clip). At 0 dB SNR, gamma 0.25 with a chosen threshold leaves 1.9% of cuts jumping over a frame on held-out clips (PHAT: 34.2%); at −10 dB 39.1% (PHAT 63.7%, ungated gamma 0.25 47.6%). Choosing gamma on other clips alone fails (9.9%) because the one clip where gamma 0.5 collapses cannot be seen from the other nine. Limits: ten clips from one gallery, synthetic delay and white noise, PSR thresholds from a probe on the same clips, and the one-frame criterion is assumed.
Paper, code and results
More in Autonomous multi-angle capture
- Does holding a fitted clock slope between audio re-syncs let free-running cameras go longer without a refresh?Does holding a fitted clock slope between audio re-syncs help? At 40 ppm it cuts out-of-tolerance time from 44% to 3.7% at 1800 s intervals, but an assumed rate wobble makes a strict line gate lock out. Synthetic audio; also corrects capture-syncbudget.
- Can a delay prior taken from a camera's own earlier windows give the oracle-prior gain in GCC audio sync?Can a delay prior from a camera's own earlier windows replace the oracle? Not yet: 22.5% to 37% wrong on the failing clips against 0% for the oracle, and an early alias becomes an hour-long lock-in.
- Does telling a learned view-synthesis blend which source is wrong help, or does it need to choose between the sources?Does telling the blend model which of two sources is wrong help, or does it need to choose between them? New occlusion and cross-check features add 0.03 dB; a per-pixel source weight adds 0.47 dB (40 of 40 test configs), recovering 0.54 of the headroom to the best blend.
- How long a lens can a 360 camera fake by cropping into a reel, and what perspective can no crop fake?How long a lens can a 360 camera fake by cropping into a reel? An 8K equirect is native only to ~24 mm-equivalent at 1080x1920 (85 mm is a 3.5x upscale), and perspective is set by distance: a stick at 0.8 m enlarges a 10 cm-nearer feature 14% vs 7.6% at an 85 mm framing distance, so portrait looks need a camera standing back. Pixel-count upper bound. First project of track 13.
- What should a travel creator pack to film from many angles under a weight budget and the 100 Wh battery limit?What should a travel creator pack to film from many angles within a weight budget and the 100 Wh battery limit? Exact search over 17,280 kits reaches all 18 band x viewpoint shots at 5 kg; a greedy packer stalls at 12 because lenses need their body (assumed device figures).
- How many cameras, and where, to film a person from every angle?How many cameras, and where, to film a person from every angle? A 30° gap between neighbours needs 13 cameras on a ring (15° needs 25), but depth error is set by convergence angle, not count: a pair's error falls 11× from 5° to 90° (7.3 to 0.64 mm) while 3 vs 16 ring cameras only halves it. Idealised 2-D lower bounds.
All 34 Autonomous multi-angle capture projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.