Does a soft confidence weight beat a hard switch for repairing stereo-depth view synthesis, and does matching cost help?
research/capture-softgate
Does a soft confidence weight beat a hard switch for repairing stereo-depth view synthesis, and does the matcher's cost help? A ramped weight on warp disagreement beats the hard gate by 0.1 to 0.5 dB in every condition (29.5 vs 29.3 dB held out at 15°) and leaves a quarter fewer seams, closing only a tenth of the oracle gap; matching cost is a worse signal than disagreement (level with hole filling, worse with a bystander). Toy scene.
What it found
A ramped weight on how much the nearest raw camera replaces a stereo-depth synthesis, driven by disagreement between the two warps, beats the hard switch in every condition tried by 0.1–0.5 dB (29.5 against 29.3 dB at a 15° gap on held-out seeds) and leaves about a quarter fewer false edges, but closes only about a tenth of the 2.6 dB gap to a per-pixel oracle. The stereo matching cost is a worse confidence signal than warp disagreement: gating on it is level with plain hole filling and worse under a bystander or sensor noise. Stylised toy scene, parameters tuned on development seeds.
A figure from its own output
Paper, code and results
More in Autonomous multi-angle capture
- How much footage does a multi-camera travel kit produce, and what limits carrying and backing it up?How much footage does a multi-camera travel kit produce, and what limits carrying and backing it up? Four cameras make ~860 GB a day; cards weigh ~8 g but the evening copy takes 2.7 h on one reader, so a safe plan needs 6 readers and ~450 g (assumed device figures).
- How often must free-running cameras be re-synced by audio, and does rejecting wild estimates help?How often must free-running cameras be re-synced by audio? At 40 ppm a sync lasts 1000 s for a 40 ms tolerance; a drift gate on fresh estimates helps only if you re-sync every 300 s. Synthetic audio.
- Does a residual audio-sync error show at the cuts of an automatic multi-camera edit?Does a residual audio-sync error show at the cuts of an automatic edit? A sync made once leaves 50% of cuts jumping over a frame; a robust fit leaves none down to -10 dB SNR (2 s windows) and then fails outright, with 11 to 24% of cuts repeating or dropping a loud event.
- Does a robust fit or a Kalman filter hold a camera's clock better between audio re-syncs than a least-squares line, once the clock drift comes from a real crystal curve?Does a robust fit or a Kalman filter hold a camera clock better between audio re-syncs? On a clock rate computed from a sourced AT-cut crystal curve the plain offset hold is already fine up to 1800 s and every fit ties; only a Kalman filter tuned to an assumed 30 ppm wobble wins. Synthetic audio.
- Does finding the start offset first let short audio windows sync two drifting cameras at lower SNR?Does a coarse offset first let short windows sync drifting cameras? A 10 s window finds the offset, then 0.5 s windows hold within a frame from -12 dB SNR; plain 1 s windows fail to -5 dB. Synthetic audio.
- Does dropping doubtful sync windows, with settings chosen on other clips, keep a multi-camera edit in sync on real audio?Does a peak-sharpness gate plus settings chosen on other clips keep the edit in sync? At 0 dB yes (1.9% of cuts over a frame held-out vs 34% for PHAT); at -10 dB only partly (39% vs 64%).
All 34 Autonomous multi-angle capture projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.