Is the depth a camera rig can actually measure good enough for the in-between angle?

What it found

capture-novelview assumed independent depth noise of 2–5%. Here depth is measured instead: block-matching stereo on a rendered rectified pair, fed into the same two-view warp and scored against ray-traced truth at a 15° gap. Mean depth error falls from 14.9% at a 3 cm baseline to 2.7% at 50 cm, but coverage peaks at 91% near 20 cm (stereo cannot match occluded or flat pixels), and the synthesised view never beats cutting to the nearest camera: best 22.3 dB at 20 cm against 24.8 dB, with exact depth at 27.3 dB. Independent noise with the same mean error is optimistic by 1optimistic by 1–5 dBndash;7 dB, because real errors are correlated and missing depth leaves holes. Sensor noise of 3% already costs 3 dB. Toy scene, smooth textures, a classic matcher with no learned prior or hole filler.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Autonomous multi-angle capture

• Can the holes and correlated errors of measured stereo depth be repaired so the in-between view beats cutting to the nearest camera?Can the holes and correlated errors of measured stereo depth be repaired so in-between synthesis beats cutting to the nearest camera? A hole filler and a no-ground-truth disagreement gate that falls back to the raw nearest camera lift a 15° gap from 22.3 dB to 29.2 dB against 24.8 dB for cutting (held-out seeds 29.2); with a bystander the gate loses its edge (26.1 vs 26.3 for plain filling). Toy scene.
• Which platform can follow a travel creator through a crowd and keep the shot?Which platform can follow a creator through a crowd and keep the shot? A drone that ignores pedestrians frames 99.9% of the time but falls to 39% at 0.3 people/m² once it avoids them; a wheeled robot is stopped by step lines 42–47% of the walk, and a flat-floor one that ignores people passes within 0.5 m ~500 times an hour. 2-D simulation, assumed platform figures.
• How much of a multi-camera shoot can an automatic edit recover, and what stops it?How much of a multi-camera shoot can an automatic edit recover? At detector noise 0.3 a fixed 3 s rotation shows mean true quality 0.546, a cut-cost (Viterbi) edit 0.802 vs 0.815 on true scores, and best-score-every-frame 0.723 at 345 cuts a minute; clock sync fails first on short clips (5 s at noise 0.25: 25% over 40 ms). Synthetic quality model, no real footage.
• When does automatic blurring miss a bystander's face, and which filming platform is conspicuous?When does automatic blurring miss a bystander's face? On an assumed size-recall model a face detector leaves 5% of identifiable faces unblurred per frame, blurring the head of any detected body under 0.1%, and independent per-frame misses show a face in some frame of a 5 s shot 100% of the time; an offline whole-track blur leaves 0 to 5%. No real footage or detector.
• Does recording only when something happens save a travel creator's battery, and what does it cost in missed moments?Does recording only on a trigger save battery, and what does it miss? With 6 thirty-second moments an hour, standby with a 10 s pre-roll uses 19.4 Wh of an 80 Wh continuous day (76% less) and captures 90% of interesting seconds; deep sleep uses 7.7 Wh but captures 69%. Misses and wake latency, not the battery, set coverage. Assumed device figures, synthetic day.
• When a camera rig records only on a trigger, should every camera have its own, or should one wake the rest?When a camera rig records only on a trigger, should every camera have its own, or should one wake the rest? Four standby cameras with their own triggers use 80 Wh of 320 Wh continuous but get all four angles on only 68% of interesting seconds; one sentinel waking the others costs the same and gives 90.5%, and any-camera wake gives 100% for 92 Wh but its false-alarm energy grows with detector count. Assumed device figures.

All 34 Autonomous multi-angle capture projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.