# Arbitrary Volumetric Refocusing of Dense and Sparse Light Fields

arXiv: https://arxiv.org/abs/2502.19238 · HTML: https://mlsanity.com/paper/2502.19238/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 26 Feb 2025.

## Builds on (most cited first, 9 of 9)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Unsupervised Learning of Depth Estimation and Visual Odometry for Sparse Light Field Cameras](https://mlsanity.com/paper/2103.11322/) (2021)
- [Deep Sparse Light Field Refocusing](https://mlsanity.com/paper/2009.02582/) (2020)
- [Interactive Light Field Tilt-Shift Refocus with Generalized Shift-and-Sum](https://mlsanity.com/paper/1910.04699/) (2019)
- [Machine learning method for light field refocusing](https://mlsanity.com/paper/2103.16020/) (2021)

## Abstract (arXiv)

A four-dimensional light field (LF) captures both textural and geometrical information of a scene in contrast to a two-dimensional image that captures only the textural information of a scene. Post-capture refocusing is an exciting application of LFs enabled by the geometric information captured. Previously proposed LF refocusing methods are mostly limited to the refocusing of single planar or volumetric region of a scene corresponding to a depth range and cannot simultaneously generate in-focus and out-of-focus regions having the same depth range. In this paper, we propose an end-to-end pipeline to simultaneously refocus multiple arbitrary planar or volumetric regions of a dense or a sparse LF. We employ pixel-dependent shifts with the typical shift-and-sum method to refocus an LF. The pixel-dependent shifts enables to refocus each pixel of an LF independently. For sparse LFs, the shift-and-sum method introduces ghosting artifacts due to the spatial undersampling. We employ a deep learning model based on U-Net architecture to almost completely eliminate the ghosting artifacts. The experimental results obtained with several LF datasets confirm the effectiveness of the proposed method. In particular, sparse LFs refocused with the proposed method archive structural similarity index higher than 0.9 despite having only 20% of data compared to dense LFs.