# Spatially Varying Exposure with 2-by-2 Multiplexing: Optimality and Universality

arXiv: https://arxiv.org/abs/2306.17367 · HTML: https://mlsanity.com/paper/2306.17367/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 64 references that are in the map.
- First posted on arXiv 30 Jun 2023.

## Builds on (most cited first, 23 of 23)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing](https://mlsanity.com/paper/1912.10557/) (2019)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- [Deep High Dynamic Range Imaging with Large Foreground Motions](https://mlsanity.com/paper/1711.08937/) (2017)
- [NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results](https://mlsanity.com/paper/2106.01439/) (2021)
- [Learning Sensor Multiplexing Design through Back-propagation](https://mlsanity.com/paper/1605.07078/) (2016)
- [Dynamic Low-light Imaging with Quanta Image Sensors](https://mlsanity.com/paper/2007.08614/) (2020)
- [Passive Inter-Photon Imaging](https://mlsanity.com/paper/2104.00059/) (2021)
- [HDR Video Reconstruction with Tri-Exposure Quad-Bayer Sensors](https://mlsanity.com/paper/2103.10982/) (2021)
- [Learning Spatially Varying Pixel Exposures for Motion Deblurring](https://mlsanity.com/paper/2204.07267/) (2022)
- [What Does a One-Bit Quanta Image Sensor Offer?](https://mlsanity.com/paper/2208.10350/) (2022)
- [HDR Imaging with Quanta Image Sensors: Theoretical Limits and Optimal Reconstruction](https://mlsanity.com/paper/2011.03614/) (2020)
- [HDR Imaging with Spatially Varying Signal-to-Noise Ratios](https://mlsanity.com/paper/2303.17253/) (2023)
- [Learning a Reinforced Agent for Flexible Exposure Bracketing Selection](https://mlsanity.com/paper/2005.12536/) (2020)
- [Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imaging](https://mlsanity.com/paper/2204.09871/) (2022)
- [Exposure-Referred Signal-to-Noise Ratio for Digital Image Sensors](https://mlsanity.com/paper/2112.05817/) (2021)
- [HDRfeat: A Feature-Rich Network for High Dynamic Range Image Reconstruction](https://mlsanity.com/paper/2211.04238/) (2022)
- [On the Insensitivity of Bit Density to Read Noise in One-bit Quanta Image Sensors](https://mlsanity.com/paper/2203.06086/) (2022)

## Built on by (most cited first, 6 of 6)

- [Generative Photography: Scene-Consistent Camera Control for Realistic Text-to-Image Synthesis](https://mlsanity.com/paper/2412.02168/) (2024)
- [Quanta Video Restoration](https://mlsanity.com/paper/2410.14994/) (2024)
- [Photon Inhibition for Energy-Efficient Single-Photon Imaging](https://mlsanity.com/paper/2409.18337/) (2024)
- [iHDR: Iterative HDR Imaging with Arbitrary Number of Exposures](https://mlsanity.com/paper/2505.22971/) (2025)
- [A Hardware-Algorithm Co-Designed Framework for HDR Imaging and Dehazing in Extreme Rocket Launch Environments](https://mlsanity.com/paper/2601.08162/) (2026)
- [Generating HDR Video from SDR Video](https://mlsanity.com/paper/2605.14703/) (2026)

## Abstract (arXiv)

The advancement of new digital image sensors has enabled the design of exposure multiplexing schemes where a single image capture can have multiple exposures and conversion gains in an interlaced format, similar to that of a Bayer color filter array. In this paper, we ask the question of how to design such multiplexing schemes for adaptive high-dynamic range (HDR) imaging where the multiplexing scheme can be updated according to the scenes. We present two new findings. (i) We address the problem of design optimality. We show that given a multiplex pattern, the conventional optimality criteria based on the input/output-referred signal-to-noise ratio (SNR) of the independently measured pixels can lead to flawed decisions because it cannot encapsulate the location of the saturated pixels. We overcome the issue by proposing a new concept known as the spatially varying exposure risk (SVE-Risk) which is a pseudo-idealistic quantification of the amount of recoverable pixels. We present an efficient enumeration algorithm to select the optimal multiplex patterns. (ii) We report a design universality observation that the design of the multiplex pattern can be decoupled from the image reconstruction algorithm. This is a significant departure from the recent literature that the multiplex pattern should be jointly optimized with the reconstruction algorithm. Our finding suggests that in the context of exposure multiplexing, an end-to-end training may not be necessary.