mlsanityData to 14 Oct 2026

Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography

arXiv 2212.12324 ↗First posted 22 Dec 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Modern mobile burst photography pipelines capture and merge a short sequence of frames to recover an enhanced image, but often disregard the 3D nature of the scene they capture, treating pixel motion between images as a 2D aggregation problem. We show that in a ''long-burst'', forty-two 12-megapixel RAW frames captured in a two-second sequence, there is enough parallax information from natural hand tremor alone to recover high-quality scene depth. To this end, we devise a test-time optimization approach that fits a neural RGB-D representation to long-burst data and simultaneously estimates scene depth and camera motion. Our plane plus depth model is trained end-to-end, and performs coarse-to-fine refinement by controlling which multi-resolution volume features the network has access to at what time during training. We validate the method experimentally, and demonstrate geometrically accurate depth reconstructions with no additional hardware or separate data pre-processing and pose-estimation steps.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 44 of its 87 references that are in the map. First posted on arXiv 22 Dec 2022.

Builds on

44 of 87 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  4. 2021Vision Transformers for Dense Predictioncited by 1,803
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  8. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  9. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 724
  10. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  11. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  12. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 533
32 more
  1. 2018Pyramid Stereo Matching Networkcited by 489
  2. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 353
  3. 2021NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Imagescited by 336
  4. 2018Unprocessing Images for Learned Raw Denoisingcited by 250
  5. 2019Depth-Aware Video Frame Interpolationcited by 234
  6. 2020Consistent Video Depth Estimationcited by 220
  7. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 201
  8. 2022Monocular Dynamic View Synthesis: A Reality Checkcited by 200
  9. 2020Robust Consistent Video Depth Estimationcited by 188
  10. 2021NeRF--: Neural Radiance Fields Without Known Camera Parameterscited by 155
  11. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 145
  12. 2020Defocus Deblurring Using Dual-Pixel Datacited by 139
  13. 2020HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matchingcited by 120
  14. 2021Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Mergingcited by 104
  15. 2021Real-time Neural Radiance Caching for Path Tracingcited by 91
  16. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  17. 2019Handheld Multi-Frame Super-Resolutioncited by 85
  18. 2018Synthetic Depth-of-Field with a Single-Camera Mobile Phonecited by 71
  19. 2021Mobile Computational Photography: A Tourcited by 62
  20. 2019Handheld Mobile Photography in Very Low Lightcited by 56
  21. 2022NAN: Noise-Aware NeRFs for Burst-Denoisingcited by 48
  22. 2022Dancing under the stars: video denoising in starlightcited by 47
  23. 2020One Shot 3D Photographycited by 45
  24. 2019A megapixel time-gated SPAD image sensor for 2D and 3D imaging applicationscited by 28
  25. 2018RGB-D Object Detection and Semantic Segmentation for Autonomous Manipulation in Cluttercited by 21
  26. 2020An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologiescited by 20
  27. 2015Locally Non-rigid Registration for Mobile HDR Photographycited by 18
  28. 2018DeepLens: Shallow Depth Of Field From A Single Imagecited by 13
  29. 2021The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinementcited by 13
  30. 2013From Shading to Local Shapecited by 12
  31. 2022RGBD Object Tracking: An In-depth Reviewcited by 11
  32. 2021S3Net: A Single Stream Structure for Depth Guided Image Relightingcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 0; 2026: 2.

8 papers.

  1. 2023Neural Spline Fields for Burst Image Fusion and Layer Separationcited by 16
  2. 2024Neural Light Spheres for Implicit Image Stitching and View Synthesiscited by 6
  3. 2024Iterative approach to reconstructing neural disparity fields from light-field datacited by 2
  4. 2026Dark3R: Learning Structure from Motion in the Darkcited by 2
  5. 2024Solutions to Deepfakes: Can Camera Hardware, Cryptography, and Deep Learning Verify Real Images?cited by 1
  6. 2024Refinement of Monocular Depth Maps via Multi-View Differentiable Renderingcited by 1
  7. 2023Fine Dense Alignment of Image Bursts through Camera Pose and Depth Estimationcited by 0
  8. 2026Mining Attribute Subspaces for Efficient Fine-tuning of 3D Foundation Modelscited by 0