Can a burst of hand-shaken low-resolution frames recover detail that merging and upsampling cannot?
research/sensing-superres
Can a burst of hand-shaken low-resolution frames recover detail that merging and upsampling cannot? At 2× in bright light a least-squares reconstruction reaches 29.5 dB at 16 frames against 24.0 for merge-and-upsample and 26.6 for merge-and-deblur, so 2.9 dB is sub-pixel information; it nearly vanishes in dim light and falls to nothing at about 0.27 px of shift error. Synthetic scenes only.
What it found
Synthetic 32×32 frames from a box-averaging sensor (Poisson shot noise, 3 e− read noise), shifted by up to two pixels, reconstructed on a 2× grid by regularised least squares. In bright light it reaches 29.5 dB at 16 frames and 30.7 dB at 32, against 24.0 dB for register, average and bilinear upsample and 26.6 dB for register, average and deconvolve: 2.6 dB of the gain is deblurring the pixel box and 2.9 dB (4.0 at 32 frames) is sub-pixel information. In dim light that part is within ±0.3 dB; registration error of 0.27 LR pixel erases it. Against a sensor with 2× the linear resolution and the same total light the burst wins at 2 frames and loses from about 4; at 3× it ties at 16 frames and at 4× it wins. Synthetic scenes, translation only, no mosaic or lens blur; the reconstruction is plain Tikhonov, not a published method.
A figure from its own output
Paper, code and results
More in Sensing, imaging and 3D
- How many milliseconds of camera-to-gyro time offset double a fused attitude error, and can online calibration recover it?How many milliseconds of camera-to-gyro time offset double a fused attitude error, and can an alignment fit recover it? About 12–14 ms at 30 deg/s motion; the fit recovers it to ~1 ms with motion and fails on a still platform. Synthetic, single axis.
- How much motion does a camera-to-gyro offset fit need, and what does clock skew do to a one-off calibration?How much motion does a camera-to-gyro offset fit need, and what does clock skew do to a one-off calibration? A 20 ms offset needs about 16 deg/s rms motion to fit reliably; 1000 ppm skew leaves 0.254 deg error after a fixed-offset fit against 0.155 deg when skew is fitted too. Stylised, synthetic.
- How accurate must a scan be to rebuild a part as parametric CAD?How accurate must a scan be to rebuild a part as parametric CAD? On synthetic plate scans, RANSAC plane and circle fits give all five dimensions within 0.2 mm for noise up to 0.2 mm, but at 0.5 mm noise the error is a bias (thickness −0.20 mm) that more points do not remove, and a 0.5% scale error alone moves an 80 mm length by 0.40 mm. Synthetic, one shape.
- How many photos does a usable 3-D reconstruction need?How many photos does a usable 3-D reconstruction need? On a synthetic object with known cameras, 12 views on a full orbit triangulate every feature, 8 views 89% and 4 or fewer at most 13%; per-point error then depends on pixel noise and pose error, not view count. Triangulation stage only (synthetic).
- Does a low-resolution thermal camera find people in darkness and smoke, and what does misalignment cost?Does a low-resolution thermal camera find people in darkness and smoke, and what does misalignment cost? On synthetic scenes RGB finds 36% in bright light and none in the dark or dense smoke, thermal alone 65% everywhere (capped by warm decoys), fusion 94% in bright light and the thermal level otherwise; a 4 px (one thermal pixel) displacement costs the fused detector 19 points and 6 px erases the gain. Synthetic only.
- Which upgrade buys the most visual-inertial accuracy: a sharper camera or a better IMU?Which upgrade buys the most visual-inertial accuracy, a sharper camera or a better IMU? On a planar synthetic smoother only camera resolution moved drift (1280 px 1.2 % vs 1.7 % at 640 px); a 0.25–4× change in IMU noise did not, on 12 s paths. Stylised, not EuRoC.
All 31 Sensing, imaging and 3D projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.