# Deep Optics for Single-shot High-dynamic-range Imaging

arXiv: https://arxiv.org/abs/1908.00620 · HTML: https://mlsanity.com/paper/1908.00620/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2019.

## Built on by (most cited first, 30 of 62)

- [HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset](https://mlsanity.com/paper/2103.14943/) (2021)
- [Deep Learning for HDR Imaging: State-of-the-Art and Future Trends](https://mlsanity.com/paper/2110.10394/) (2021)
- [Curriculum Learning for ab initio Deep Learned Refractive Optics](https://mlsanity.com/paper/2302.01089/) (2023)
- [RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image](https://mlsanity.com/paper/2309.02020/) (2023)
- [PrivHAR: Recognizing Human Actions From Privacy-preserving Lens](https://mlsanity.com/paper/2206.03891/) (2022)
- [Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems](https://mlsanity.com/paper/2104.02820/) (2021)
- [The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection](https://mlsanity.com/paper/2212.04441/) (2022)
- [Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction](https://mlsanity.com/paper/2304.12372/) (2023)
- [Unidirectional Imaging using Deep Learning-Designed Materials](https://mlsanity.com/paper/2212.02025/) (2022)
- [Thin On-Sensor Nanophotonic Array Cameras](https://mlsanity.com/paper/2308.02797/) (2023)
- [Learning Spatially Varying Pixel Exposures for Motion Deblurring](https://mlsanity.com/paper/2204.07267/) (2022)
- [End-to-End Hybrid Refractive-Diffractive Lens Design with Differentiable Ray-Wave Model](https://mlsanity.com/paper/2406.00834/) (2024)
- [CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo](https://mlsanity.com/paper/2104.04641/) (2021)
- [MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding](https://mlsanity.com/paper/2112.05221/) (2021)
- [SoDaCam: Software-defined Cameras via Single-Photon Imaging](https://mlsanity.com/paper/2309.00066/) (2023)
- [Annular Computational Imaging: Capture Clear Panoramic Images through Simple Lens](https://mlsanity.com/paper/2206.06070/) (2022)
- [Task-Driven Lens Design](https://mlsanity.com/paper/2305.17185/) (2023)
- [Deep Optics for Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2404.05274/) (2024)
- [Computational Imaging and Artificial Intelligence: The Next Revolution of Mobile Vision](https://mlsanity.com/paper/2109.08880/) (2021)
- [Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems](https://mlsanity.com/paper/2404.19201/) (2024)
- [CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras](https://mlsanity.com/paper/2406.09409/) (2024)
- [Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging](https://mlsanity.com/paper/2204.09871/) (2022)
- [Diffractive Interconnects: All-Optical Permutation Operation Using Diffractive Networks](https://mlsanity.com/paper/2206.10152/) (2022)
- [Differentiable Sensor Layouts for End-to-End Learning of Task-Specific Camera Parameters](https://mlsanity.com/paper/2304.14736/) (2023)
- [DISeR: Designing Imaging Systems with Reinforcement Learning](https://mlsanity.com/paper/2309.13851/) (2023)
- [Learning Lens Blur Fields](https://mlsanity.com/paper/2310.11535/) (2023)
- [Video Reconstruction from a Single Motion Blurred Image using Learned Dynamic Phase Coding](https://mlsanity.com/paper/2112.14768/) (2021)
- [PUERT: Probabilistic Under-sampling and Explicable Reconstruction Network for CS-MRI](https://mlsanity.com/paper/2204.11189/) (2022)
- [TiDy-PSFs: Computational Imaging with Time-Averaged Dynamic Point-Spread-Functions](https://mlsanity.com/paper/2303.17583/) (2023)
- [Minimalist Vision with Freeform Pixels](https://mlsanity.com/paper/2501.00142/) (2024)

## Abstract (arXiv)

High-dynamic-range (HDR) imaging is crucial for many computer graphics and vision applications. Yet, acquiring HDR images with a single shot remains a challenging problem. Whereas modern deep learning approaches are successful at hallucinating plausible HDR content from a single low-dynamic-range (LDR) image, saturated scene details often cannot be faithfully recovered. Inspired by recent deep optical imaging approaches, we interpret this problem as jointly training an optical encoder and electronic decoder where the encoder is parameterized by the point spread function (PSF) of the lens, the bottleneck is the sensor with a limited dynamic range, and the decoder is a convolutional neural network (CNN). The lens surface is then jointly optimized with the CNN in a training phase; we fabricate this optimized optical element and attach it as a hardware add-on to a conventional camera during inference. In extensive simulations and with a physical prototype, we demonstrate that this end-to-end deep optical imaging approach to single-shot HDR imaging outperforms both purely CNN-based approaches and other PSF engineering approaches.