# Light field Rectification based on relative pose estimation

arXiv: https://arxiv.org/abs/2201.12533 · HTML: https://mlsanity.com/paper/2201.12533/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 60 references that are in the map.
- First posted on arXiv 29 Jan 2022.

## Builds on (most cited first, 5 of 5)

- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization](https://mlsanity.com/paper/2004.02215/) (2020)
- [LiFF: Light Field Features in Scale and Depth](https://mlsanity.com/paper/1901.03916/) (2019)
- [High-Order Residual Network for Light Field Super-Resolution](https://mlsanity.com/paper/2003.13094/) (2020)
- [Texture-enhanced Light Field Super-resolution with Spatio-Angular Decomposition Kernels](https://mlsanity.com/paper/2111.04069/) (2021)

## Abstract (arXiv)

Hand-held light field (LF) cameras have unique advantages in computer vision such as 3D scene reconstruction and depth estimation. However, the related applications are limited by the ultra-small baseline, e.g., leading to the extremely low depth resolution in reconstruction. To solve this problem, we propose to rectify LF to obtain a large baseline. Specifically, the proposed method aligns two LFs captured by two hand-held LF cameras with a random relative pose, and extracts the corresponding row-aligned sub-aperture images (SAIs) to obtain an LF with a large baseline. For an accurate rectification, a method for pose estimation is also proposed, where the relative rotation and translation between the two LF cameras are estimated. The proposed pose estimation minimizes the degree of freedom (DoF) in the LF-point-LF-point correspondence model and explicitly solves this model in a linear way. The proposed pose estimation outperforms the state-of-the-art algorithms by providing more accurate results to support rectification. The significantly improved depth resolution in 3D reconstruction demonstrates the effectiveness of the proposed LF rectification.