# Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselines

arXiv: https://arxiv.org/abs/2108.07408 · HTML: https://mlsanity.com/paper/2108.07408/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 39 references that are in the map.
- First posted on arXiv 17 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images](https://mlsanity.com/paper/1804.02379/) (2018)
- [Light Field Spatial Super-resolution via Deep Combinatorial Geometry Embedding and Structural Consistency Regularization](https://mlsanity.com/paper/2004.02215/) (2020)
- [Accurate Light Field Depth Estimation with Superpixel Regularization over Partially Occluded Regions](https://mlsanity.com/paper/1708.01964/) (2017)
- [Light Field Reconstruction Using Shearlet Transform](https://mlsanity.com/paper/1509.08969/) (2015)
- [Learning Light Field Angular Super-Resolution via a Geometry-Aware Network](https://mlsanity.com/paper/2002.11263/) (2020)
- [Towards a quality metric for dense light fields](https://mlsanity.com/paper/1704.07576/) (2017)
- [Deep Spatial-angular Regularization for Compressive Light Field Reconstruction over Coded Apertures](https://mlsanity.com/paper/2007.11882/) (2020)

## Built on by (most cited first, 3 of 3)

- [Occlusion-Aware Cost Constructor for Light Field Depth Estimation](https://mlsanity.com/paper/2203.01576/) (2022)
- [Content-aware Warping for View Synthesis](https://mlsanity.com/paper/2201.09023/) (2022)
- [Geo-NI: Geometry-aware Neural Interpolation for Light Field Rendering](https://mlsanity.com/paper/2206.09736/) (2022)

## Abstract (arXiv)

In this paper, we tackle the problem of dense light field (LF) reconstruction from sparsely-sampled ones with wide baselines and propose a learnable model, namely dynamic interpolation, to replace the commonly-used geometry warping operation. Specifically, with the estimated geometric relation between input views, we first construct a lightweight neural network to dynamically learn weights for interpolating neighbouring pixels from input views to synthesize each pixel of novel views independently. In contrast to the fixed and content-independent weights employed in the geometry warping operation, the learned interpolation weights implicitly incorporate the correspondences between the source and novel views and adapt to different image content information. Then, we recover the spatial correlation between the independently synthesized pixels of each novel view by referring to that of input views using a geometry-based spatial refinement module. We also constrain the angular correlation between the novel views through a disparity-oriented LF structure loss. Experimental results on LF datasets with wide baselines show that the reconstructed LFs achieve much higher PSNR/SSIM and preserve the LF parallax structure better than state-of-the-art methods. The source code is publicly available at https://github.com/MantangGuo/DI4SLF.