# Joint 3D Shape and Motion Estimation from Rolling Shutter Light-Field Images

arXiv: https://arxiv.org/abs/2311.01292 · HTML: https://mlsanity.com/paper/2311.01292/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 42 references that are in the map.
- First posted on arXiv 2 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AI](https://mlsanity.com/paper/2109.08238/) (2021)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images](https://mlsanity.com/paper/1804.02379/) (2018)
- [Occlusion-Aware Cost Constructor for Light Field Depth Estimation](https://mlsanity.com/paper/2203.01576/) (2022)
- [SUNet: Symmetric Undistortion Network for Rolling Shutter Correction](https://mlsanity.com/paper/2108.04775/) (2021)
- [LiFF: Light Field Features in Scale and Depth](https://mlsanity.com/paper/1901.03916/) (2019)
- [Towards a quality metric for dense light fields](https://mlsanity.com/paper/1704.07576/) (2017)
- [Light Field Blind Motion Deblurring](https://mlsanity.com/paper/1704.05416/) (2017)
- [A Generic Multi-Projection-Center Model and Calibration Method for Light Field Cameras](https://mlsanity.com/paper/1808.02244/) (2018)

## Built on by (most cited first, 3 of 3)

- [Order-One Rolling Shutter Cameras](https://mlsanity.com/paper/2403.11295/) (2024)
- [Dense Scene Reconstruction from Light-Field Images Affected by Rolling Shutter](https://mlsanity.com/paper/2412.03518/) (2024)
- [Single-View Rolling-Shutter SfM](https://mlsanity.com/paper/2603.11888/) (2026)

## Abstract (arXiv)

In this paper, we propose an approach to address the problem of 3D reconstruction of scenes from a single image captured by a light-field camera equipped with a rolling shutter sensor. Our method leverages the 3D information cues present in the light-field and the motion information provided by the rolling shutter effect. We present a generic model for the imaging process of this sensor and a two-stage algorithm that minimizes the re-projection error while considering the position and motion of the camera in a motion-shape bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge, this is the first study to leverage this type of sensor for this purpose. We also present a new benchmark dataset composed of different light-fields showing rolling shutter effects, which can be used as a common base to improve the evaluation and tracking the progress in the field. We demonstrate the effectiveness and advantages of our approach through several experiments conducted for different scenes and types of motions. The source code and dataset are publicly available at: https://github.com/ICB-Vision-AI/RSLF