# Acquiring a Dynamic Light Field through a Single-Shot Coded Image

arXiv: https://arxiv.org/abs/2204.12089 · HTML: https://mlsanity.com/paper/2204.12089/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 59 references that are in the map.
- First posted on arXiv 26 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines](https://mlsanity.com/paper/1905.00889/) (2019)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [SynSin: End-to-end View Synthesis from a Single Image](https://mlsanity.com/paper/1912.08804/) (2019)
- [Single-View View Synthesis with Multiplane Images](https://mlsanity.com/paper/2004.11364/) (2020)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [3D Ken Burns Effect from a Single Image](https://mlsanity.com/paper/1909.05483/) (2019)
- [Plug-and-Play Algorithms for Large-scale Snapshot Compressive Imaging](https://mlsanity.com/paper/2003.13654/) (2020)
- [Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Image](https://mlsanity.com/paper/2012.09854/) (2020)
- [Learning to Synthesize a 4D RGBD Light Field from a Single Image](https://mlsanity.com/paper/1708.03292/) (2017)
- [EPINET: A Fully-Convolutional Neural Network Using Epipolar Geometry for Depth from Light Field Images](https://mlsanity.com/paper/1804.02379/) (2018)
- [Learning Sensor Multiplexing Design through Back-propagation](https://mlsanity.com/paper/1605.07078/) (2016)
- [A 4D Light-Field Dataset and CNN Architectures for Material Recognition](https://mlsanity.com/paper/1608.06985/) (2016)
- [DeepBinaryMask: Learning a Binary Mask for Video Compressive Sensing](https://mlsanity.com/paper/1607.03343/) (2016)
- [Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems](https://mlsanity.com/paper/2104.02820/) (2021)
- [Light Field Video Capture Using a Learning-Based Hybrid Imaging System](https://mlsanity.com/paper/1705.02997/) (2017)
- [Deep Spatial-angular Regularization for Compressive Light Field Reconstruction over Coded Apertures](https://mlsanity.com/paper/2007.11882/) (2020)
- [Learning a Probabilistic Strategy for Computational Imaging Sensor Selection](https://mlsanity.com/paper/2003.10424/) (2020)
- [A Unified Learning Based Framework for Light Field Reconstruction from Coded Projections](https://mlsanity.com/paper/1812.10532/) (2018)
- [Fast and Accurate Reconstruction of Compressed Color Light Field](https://mlsanity.com/paper/1801.10351/) (2018)
- [Compressive Light Field Reconstructions using Deep Learning](https://mlsanity.com/paper/1802.01722/) (2018)

## Built on by (most cited first, 4 of 4)

- [Time-Efficient Light-Field Acquisition Using Coded Aperture and Events](https://mlsanity.com/paper/2403.07244/) (2024)
- [V2V3D: View-to-View Denoised 3D Reconstruction for Light-Field Microscopy](https://mlsanity.com/paper/2504.07853/) (2025)
- [Coded-E2LF: Coded Aperture Light Field Imaging from Events](https://mlsanity.com/paper/2602.22620/) (2026)
- [Reconstructing Continuous Light Field From Single Coded Image](https://mlsanity.com/paper/2311.09646/) (2023)

## Abstract (arXiv)

We propose a method for compressively acquiring a dynamic light field (a 5-D volume) through a single-shot coded image (a 2-D measurement). We designed an imaging model that synchronously applies aperture coding and pixel-wise exposure coding within a single exposure time. This coding scheme enables us to effectively embed the original information into a single observed image. The observed image is then fed to a convolutional neural network (CNN) for light-field reconstruction, which is jointly trained with the camera-side coding patterns. We also developed a hardware prototype to capture a real 3-D scene moving over time. We succeeded in acquiring a dynamic light field with 5x5 viewpoints over 4 temporal sub-frames (100 views in total) from a single observed image. Repeating capture and reconstruction processes over time, we can acquire a dynamic light field at 4x the frame rate of the camera. To our knowledge, our method is the first to achieve a finer temporal resolution than the camera itself in compressive light-field acquisition. Our software is available from our project webpage