# HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset

arXiv: https://arxiv.org/abs/2103.14943 · HTML: https://mlsanity.com/paper/2103.14943/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 63 references that are in the map.
- First posted on arXiv 27 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- [Deep High Dynamic Range Imaging with Large Foreground Motions](https://mlsanity.com/paper/1711.08937/) (2017)
- [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018)
- [HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions](https://mlsanity.com/paper/2007.01628/) (2020)
- [Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss](https://mlsanity.com/paper/2005.07335/) (2020)
- [Deep Optics for Single-shot High-dynamic-range Imaging](https://mlsanity.com/paper/1908.00620/) (2019)
- [Handheld Mobile Photography in Very Low Light](https://mlsanity.com/paper/1910.11336/) (2019)
- [Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications](https://mlsanity.com/paper/1904.11176/) (2019)
- [Learning High Dynamic Range from Outdoor Panoramas](https://mlsanity.com/paper/1703.10200/) (2017)
- [JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video](https://mlsanity.com/paper/1909.04391/) (2019)
- [A Unified Framework for Multi-Sensor HDR Video Reconstruction](https://mlsanity.com/paper/1308.4908/) (2013)
- [A Maximum A Posteriori Estimation Framework for Robust High Dynamic Range Video Synthesis](https://mlsanity.com/paper/1612.02761/) (2016)
- [Hybrid Loss for Learning Single-Image-based HDR Reconstruction](https://mlsanity.com/paper/1812.07134/) (2018)

## Built on by (most cited first, 30 of 45)

- [Deep Learning for HDR Imaging: State-of-the-Art and Future Trends](https://mlsanity.com/paper/2110.10394/) (2021)
- [HDR-Plenoxels: Self-Calibrating High Dynamic Range Radiance Fields](https://mlsanity.com/paper/2208.06787/) (2022)
- [HDRFlow: Real-Time HDR Video Reconstruction with Large Motions](https://mlsanity.com/paper/2403.03447/) (2024)
- [Text2Light: Zero-Shot Text-Driven HDR Panorama Generation](https://mlsanity.com/paper/2209.09898/) (2022)
- [LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction](https://mlsanity.com/paper/2308.11116/) (2023)
- [Towards Real-World HDR Video Reconstruction: A Large-Scale Benchmark Dataset and A Two-Stage Alignment Network](https://mlsanity.com/paper/2405.00244/) (2024)
- [How to cheat with metrics in single-image HDR reconstruction](https://mlsanity.com/paper/2108.08713/) (2021)
- [Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models](https://mlsanity.com/paper/2303.13031/) (2023)
- [Inverting the Imaging Process by Learning an Implicit Camera Model](https://mlsanity.com/paper/2304.12748/) (2023)
- [Self-Supervised High Dynamic Range Imaging with Multi-Exposure Images in Dynamic Scenes](https://mlsanity.com/paper/2310.01840/) (2023)
- [A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking](https://mlsanity.com/paper/2505.19858/) (2025)
- [Pano-NeRF: Synthesizing High Dynamic Range Novel Views with Geometry from Sparse Low Dynamic Range Panoramic Images](https://mlsanity.com/paper/2312.15942/) (2023)
- [LHDR: HDR Reconstruction for Legacy Content using a Lightweight DNN](https://mlsanity.com/paper/2211.11270/) (2022)
- [Unsupervised HDR Image and Video Tone Mapping via Contrastive Learning](https://mlsanity.com/paper/2303.07327/) (2023)
- [HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains](https://mlsanity.com/paper/2304.04773/) (2023)
- [Diffusion-Promoted HDR Video Reconstruction](https://mlsanity.com/paper/2406.08204/) (2024)
- [Self-supervised HDR Imaging from Motion and Exposure Cues](https://mlsanity.com/paper/2203.12311/) (2022)
- [Joint Super-Resolution and Inverse Tone-Mapping: A Feature Decomposition Aggregation Network and A New Benchmark](https://mlsanity.com/paper/2207.03367/) (2022)
- [DeepHS-HDRVideo: Deep High Speed High Dynamic Range Video Reconstruction](https://mlsanity.com/paper/2210.04429/) (2022)
- [Efficient HDR Reconstruction from Real-World Raw Images](https://mlsanity.com/paper/2306.10311/) (2023)
- [HDR Imaging for Dynamic Scenes with Events](https://mlsanity.com/paper/2404.03210/) (2024)
- [Capturing Stable HDR Videos Using a Dual-Camera System](https://mlsanity.com/paper/2507.06593/) (2025)
- [AdaptiveAE: An Adaptive Exposure Strategy for HDR Capturing in Dynamic Scenes](https://mlsanity.com/paper/2508.13503/) (2025)
- [DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models](https://mlsanity.com/paper/2604.06161/) (2026)
- [Unsupervised HDR Imaging: What Can Be Learned from a Single 8-bit Video?](https://mlsanity.com/paper/2202.05522/) (2022)
- [SJ-HD^2R: Selective Joint High Dynamic Range and Denoising Imaging for Dynamic Scenes](https://mlsanity.com/paper/2206.09611/) (2022)
- [Examining Autoexposure for Challenging Scenes](https://mlsanity.com/paper/2309.04542/) (2023)
- [Dual Exposure Stereo for Extended Dynamic Range 3D Imaging](https://mlsanity.com/paper/2412.02351/) (2024)
- [S2R-HDR: A Large-Scale Rendered Dataset for HDR Fusion](https://mlsanity.com/paper/2504.07667/) (2025)
- [Mono4DGS-HDR: High Dynamic Range 4D Gaussian Splatting from Alternating-exposure Monocular Videos](https://mlsanity.com/paper/2510.18489/) (2025)

## Abstract (arXiv)

High dynamic range (HDR) video reconstruction from sequences captured with alternating exposures is a very challenging problem. Existing methods often align low dynamic range (LDR) input sequence in the image space using optical flow, and then merge the aligned images to produce HDR output. However, accurate alignment and fusion in the image space are difficult due to the missing details in the over-exposed regions and noise in the under-exposed regions, resulting in unpleasing ghosting artifacts. To enable more accurate alignment and HDR fusion, we introduce a coarse-to-fine deep learning framework for HDR video reconstruction. Firstly, we perform coarse alignment and pixel blending in the image space to estimate the coarse HDR video. Secondly, we conduct more sophisticated alignment and temporal fusion in the feature space of the coarse HDR video to produce better reconstruction. Considering the fact that there is no publicly available dataset for quantitative and comprehensive evaluation of HDR video reconstruction methods, we collect such a benchmark dataset, which contains 97 sequences of static scenes and 184 testing pairs of dynamic scenes. Extensive experiments show that our method outperforms previous state-of-the-art methods. Our dataset, code and model will be made publicly available.