# Pix2HDR -- A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos

arXiv: https://arxiv.org/abs/2310.16139 · HTML: https://mlsanity.com/paper/2310.16139/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 48 references that are in the map.
- First posted on arXiv 24 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Learning to Extract a Video Sequence from a Single Motion-Blurred Image](https://mlsanity.com/paper/1804.04065/) (2018)
- [Bringing Alive Blurred Moments](https://mlsanity.com/paper/1804.02913/) (2018)
- [HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset](https://mlsanity.com/paper/2103.14943/) (2021)
- [Learning to Generate Images with Perceptual Similarity Metrics](https://mlsanity.com/paper/1511.06409/) (2015)
- [HDR Video Reconstruction with Tri-Exposure Quad-Bayer Sensors](https://mlsanity.com/paper/2103.10982/) (2021)
- [DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction](https://mlsanity.com/paper/2210.04429/) (2022)

## Built on by (most cited first, 1 of 1)

- [Capturing Stable HDR Videos Using a Dual-Camera System](https://mlsanity.com/paper/2507.06593/) (2025)

## Abstract (arXiv)

Accurately capturing dynamic scenes with wide-ranging motion and light intensity is crucial for many vision applications. However, acquiring high-speed high dynamic range (HDR) video is challenging because the camera's frame rate restricts its dynamic range. Existing methods sacrifice speed to acquire multi-exposure frames. Yet, misaligned motion in these frames can still pose complications for HDR fusion algorithms, resulting in artifacts. Instead of frame-based exposures, we sample the videos using individual pixels at varying exposures and phase offsets. Implemented on a monochrome pixel-wise programmable image sensor, our sampling pattern simultaneously captures fast motion at a high dynamic range. We then transform pixel-wise outputs into an HDR video using end-to-end learned weights from deep neural networks, achieving high spatiotemporal resolution with minimized motion blurring. We demonstrate aliasing-free HDR video acquisition at 1000 FPS, resolving fast motion under low-light conditions and against bright backgrounds - both challenging conditions for conventional cameras. By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.