# Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems

arXiv: https://arxiv.org/abs/2104.02820 · HTML: https://mlsanity.com/paper/2104.02820/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 66 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Learning-Based View Synthesis for Light Field Cameras](https://mlsanity.com/paper/1609.02974/) (2016)
- [Deep Optics for Monocular Depth Estimation and 3D Object Detection](https://mlsanity.com/paper/1904.08601/) (2019)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Learning Sensor Multiplexing Design through Back-propagation](https://mlsanity.com/paper/1605.07078/) (2016)
- [Data-Driven Design for Fourier Ptychographic Microscopy](https://mlsanity.com/paper/1904.04175/) (2019)
- [Deep Spatial-angular Regularization for Compressive Light Field Reconstruction over Coded Apertures](https://mlsanity.com/paper/2007.11882/) (2020)

## Built on by (most cited first, 15 of 15)

- [Image sensing with multilayer, nonlinear optical neural networks](https://mlsanity.com/paper/2207.14293/) (2022)
- [Learning Spatially Varying Pixel Exposures for Motion Deblurring](https://mlsanity.com/paper/2204.07267/) (2022)
- [Time-Efficient Light-Field Acquisition Using Coded Aperture and Events](https://mlsanity.com/paper/2403.07244/) (2024)
- [Deep Optics for Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2404.05274/) (2024)
- [Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision](https://mlsanity.com/paper/2109.08880/) (2021)
- [Deep Optical Coding Design in Computational Imaging](https://mlsanity.com/paper/2207.00164/) (2022)
- [Acquiring a Dynamic Light Field through a Single-Shot Coded Image](https://mlsanity.com/paper/2204.12089/) (2022)
- [Depth Estimation from a Single Optical Encoded Image using a Learned Colored-Coded Aperture](https://mlsanity.com/paper/2309.08033/) (2023)
- [Learning to See Through Dazzle](https://mlsanity.com/paper/2402.15919/) (2024)
- [Learning rich optical embeddings for privacy-preserving lensless image classification](https://mlsanity.com/paper/2206.01429/) (2022)
- [Coded-E2LF: Coded Aperture Light Field Imaging from Events](https://mlsanity.com/paper/2602.22620/) (2026)
- [Fast Disparity Estimation from a Single Compressed Light Field Measurement](https://mlsanity.com/paper/2209.11342/) (2022)
- [Reconstructing Continuous Light Field From Single Coded Image](https://mlsanity.com/paper/2311.09646/) (2023)
- [Solving Vision Tasks with Simple Photoreceptors Instead of Cameras](https://mlsanity.com/paper/2406.11769/) (2024)
- [Lumosaic: Hyperspectral Video via Active Illumination and Coded-Exposure Pixels](https://mlsanity.com/paper/2602.22140/) (2026)

## Abstract (arXiv)

Compressive imaging using coded apertures (CA) is a powerful technique that can be used to recover depth, light fields, hyperspectral images and other quantities from a single snapshot. The performance of compressive imaging systems based on CAs mostly depends on two factors: the properties of the mask's attenuation pattern, that we refer to as "codification" and the computational techniques used to recover the quantity of interest from the coded snapshot. In this work, we introduce the idea of using time-varying CAs synchronized with spatially varying pixel shutters. We divide the exposure of a sensor into sub-exposures at the beginning of which the CA mask changes and at which the sensor's pixels are simultaneously and individually switched "on" or "off". This is a practically appealing codification as it does not introduce additional optical components other than the already present CA but uses a change in the pixel shutter that can be easily realized electronically. We show that our proposed time multiplexed coded aperture (TMCA) can be optimized end-to-end and induces better coded snapshots enabling superior reconstructions in two different applications: compressive light field imaging and hyperspectral imaging. We demonstrate both in simulation and on real captures (taken with prototypes we built) that this codification outperforms the state-of-the-art compressive imaging systems by more than 4dB in those applications.