# MantissaCam: Learning Snapshot High-dynamic-range Imaging with Perceptually-based In-pixel Irradiance Encoding

arXiv: https://arxiv.org/abs/2112.05221 · HTML: https://mlsanity.com/paper/2112.05221/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 77 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018)
- [Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss](https://mlsanity.com/paper/2005.07335/) (2020)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [High Flux Passive Imaging with Single-Photon Sensors](https://mlsanity.com/paper/1902.10190/) (2019)
- [On Unlimited Sampling and Reconstruction](https://mlsanity.com/paper/1905.03901/) (2019)
- [Convolutional Sparse Coding for High Dynamic Range Imaging](https://mlsanity.com/paper/1806.04942/) (2018)
- [Robust Multi-Image HDR Reconstruction for the Modulo Camera](https://mlsanity.com/paper/1707.01317/) (2017)
- [Signal Reconstruction from Modulo Observations](https://mlsanity.com/paper/1812.00557/) (2018)
- [Reconstruction from Periodic Nonlinearities, With Applications to HDR Imaging](https://mlsanity.com/paper/1710.00109/) (2017)

## Built on by (most cited first, 10 of 10)

- [Learning Spatially Varying Pixel Exposures for Motion Deblurring](https://mlsanity.com/paper/2204.07267/) (2022)
- [PixelRNN: In-pixel Recurrent Neural Networks for End-to-end-optimized Perception with Neural Sensors](https://mlsanity.com/paper/2304.05440/) (2023)
- [High Dynamic Range Modulo Imaging for Robust Object Detection in Autonomous Driving](https://mlsanity.com/paper/2504.11472/) (2025)
- [Autoregressive High-Order Finite Difference Modulo Imaging: High-Dynamic Range for Computer Vision Applications](https://mlsanity.com/paper/2504.04228/) (2025)
- [Neural Ganglion Sensors: Learning Task-specific Event Cameras Inspired by the Neural Circuit of the Human Retina](https://mlsanity.com/paper/2504.13457/) (2025)
- [Single-shot HDR using conventional image sensor shutter functions and optical randomization](https://mlsanity.com/paper/2506.22426/) (2025)
- [Snapshot High Dynamic Range Imaging with a Polarization Camera](https://mlsanity.com/paper/2308.08094/) (2023)
- [Deep Lightweight Unrolled Network for High Dynamic Range Modulo Imaging](https://mlsanity.com/paper/2601.12526/) (2026)
- [Lucky High Dynamic Range Smartphone Imaging](https://mlsanity.com/paper/2604.19976/) (2026)
- [Single-Shot HDR Recovery via a Video Diffusion Prior](https://mlsanity.com/paper/2605.11628/) (2026)

## Abstract (arXiv)

The ability to image high-dynamic-range (HDR) scenes is crucial in many computer vision applications. The dynamic range of conventional sensors, however, is fundamentally limited by their well capacity, resulting in saturation of bright scene parts. To overcome this limitation, emerging sensors offer in-pixel processing capabilities to encode the incident irradiance. Among the most promising encoding schemes is modulo wrapping, which results in a computational photography problem where the HDR scene is computed by an irradiance unwrapping algorithm from the wrapped low-dynamic-range (LDR) sensor image. Here, we design a neural network--based algorithm that outperforms previous irradiance unwrapping methods and we design a perceptually inspired "mantissa" encoding scheme that more efficiently wraps an HDR scene into an LDR sensor. Combined with our reconstruction framework, MantissaCam achieves state-of-the-art results among modulo-type snapshot HDR imaging approaches. We demonstrate the efficacy of our method in simulation and show benefits of our algorithm on modulo images captured with a prototype implemented with a programmable sensor.