# A Generative Model for Generic Light Field Reconstruction

arXiv: https://arxiv.org/abs/2005.06508 · HTML: https://mlsanity.com/paper/2005.06508/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 May 2020.

## Built on by (most cited first, 4 of 4)

- [Regularising Inverse Problems with Generative Machine Learning Models](https://mlsanity.com/paper/2107.11191/) (2021)
- [Light Field Implicit Representation for Flexible Resolution Reconstruction](https://mlsanity.com/paper/2112.00185/) (2021)
- [Efficient Light Field Reconstruction via Spatio-Angular Dense Network](https://mlsanity.com/paper/2108.03635/) (2021)
- [Spectral Reconstruction and Disparity from Spatio-Spectrally Coded Light Fields via Multi-Task Deep Learning](https://mlsanity.com/paper/2103.10179/) (2021)

## Abstract (arXiv)

Recently deep generative models have achieved impressive progress in modeling the distribution of training data. In this work, we present for the first time a generative model for 4D light field patches using variational autoencoders to capture the data distribution of light field patches. We develop a generative model conditioned on the central view of the light field and incorporate this as a prior in an energy minimization framework to address diverse light field reconstruction tasks. While pure learning-based approaches do achieve excellent results on each instance of such a problem, their applicability is limited to the specific observation model they have been trained on. On the contrary, our trained light field generative model can be incorporated as a prior into any model-based optimization approach and therefore extend to diverse reconstruction tasks including light field view synthesis, spatial-angular super resolution and reconstruction from coded projections. Our proposed method demonstrates good reconstruction, with performance approaching end-to-end trained networks, while outperforming traditional model-based approaches on both synthetic and real scenes. Furthermore, we show that our approach enables reliable light field recovery despite distortions in the input.