# Towards Real-World HDR Video Reconstruction: A Large-Scale Benchmark Dataset and A Two-Stage Alignment Network

arXiv: https://arxiv.org/abs/2405.00244 · HTML: https://mlsanity.com/paper/2405.00244/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 23 of 23)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [High Speed and High Dynamic Range Video with an Event Camera](https://mlsanity.com/paper/1906.07165/) (2019)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- [Deep High Dynamic Range Imaging with Large Foreground Motions](https://mlsanity.com/paper/1711.08937/) (2017)
- [Ghost-free High Dynamic Range Imaging with Context-aware Transformer](https://mlsanity.com/paper/2208.05114/) (2022)
- [HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization](https://mlsanity.com/paper/2105.13084/) (2021)
- [ADNet: Attention-guided Deformable Convolutional Network for High Dynamic Range Imaging](https://mlsanity.com/paper/2105.10697/) (2021)
- [HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset](https://mlsanity.com/paper/2103.14943/) (2021)
- [Alignment-free HDR Deghosting with Semantics Consistent Transformer](https://mlsanity.com/paper/2305.18135/) (2023)
- [RawHDR: High Dynamic Range Image Reconstruction from a Single Raw Image](https://mlsanity.com/paper/2309.02020/) (2023)
- [HDRFlow: Real-Time HDR Video Reconstruction with Large Motions](https://mlsanity.com/paper/2403.03447/) (2024)
- [Motion Basis Learning for Unsupervised Deep Homography Estimation with Subspace Projection](https://mlsanity.com/paper/2103.15346/) (2021)
- [A Unified Framework for Multi-Sensor HDR Video Reconstruction](https://mlsanity.com/paper/1308.4908/) (2013)
- [Single-Image HDR Reconstruction by Multi-Exposure Generation](https://mlsanity.com/paper/2210.15897/) (2022)
- [Locally Non-rigid Registration for Mobile HDR Photography](https://mlsanity.com/paper/1504.01441/) (2015)
- [LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction](https://mlsanity.com/paper/2308.11116/) (2023)
- [FlexHDR: Modelling Alignment and Exposure Uncertainties for Flexible HDR Imaging](https://mlsanity.com/paper/2201.02625/) (2022)
- [A Unified HDR Imaging Method with Pixel and Patch Level](https://mlsanity.com/paper/2304.06943/) (2023)
- [Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models](https://mlsanity.com/paper/2303.13031/) (2023)
- [HDR Video Reconstruction with a Large Dynamic Dataset in Raw and sRGB Domains](https://mlsanity.com/paper/2304.04773/) (2023)
- [User Generated HDR Gaming Video Streaming: Dataset, Codec Comparison and Challenges](https://mlsanity.com/paper/2103.02189/) (2021)

## Built on by (most cited first, 14 of 14)

- [UltraFusion: Ultra High Dynamic Imaging using Exposure Fusion](https://mlsanity.com/paper/2501.11515/) (2025)
- [A Unified Solution to Video Fusion: From Multi-Frame Learning to Benchmarking](https://mlsanity.com/paper/2505.19858/) (2025)
- [Capturing Stable HDR Videos Using a Dual-Camera System](https://mlsanity.com/paper/2507.06593/) (2025)
- [AdaptiveAE: An Adaptive Exposure Strategy for HDR Capturing in Dynamic Scenes](https://mlsanity.com/paper/2508.13503/) (2025)
- [S2R-HDR: A Large-Scale Rendered Dataset for HDR Fusion](https://mlsanity.com/paper/2504.07667/) (2025)
- [F2HDR: Two-Stage HDR Video Reconstruction via Flow Adapter and Physical Motion Modeling](https://mlsanity.com/paper/2603.14920/) (2026)
- [HDR Video Generation via Latent Alignment with Logarithmic Encoding](https://mlsanity.com/paper/2604.11788/) (2026)
- [High Dynamic Range Video Compression: A Large-Scale Benchmark Dataset and A Learned Bit-depth Scalable Compression Algorithm](https://mlsanity.com/paper/2503.00410/) (2025)
- [Wavelet-Domain Masked Image Modeling for Color-Consistent HDR Video Reconstruction](https://mlsanity.com/paper/2602.07393/) (2026)
- [InstantHDR: Single-forward Gaussian Splatting Initialization for HDR 3D Reconstruction](https://mlsanity.com/paper/2603.11298/) (2026)
- [ExpoMotion: A Large-Scale Benchmark and A Householder Projection Network for Multi-Exposure Fusion](https://mlsanity.com/paper/2607.03110/) (2026)
- [Coarse-to-fine Framework for Generative MEF via Implicit Neural Representation](https://mlsanity.com/paper/2607.17611/) (2026)
- [Depth-guided Multi-view Exposure Bracketing for HDR Robot Vision](https://mlsanity.com/paper/2608.16014/) (2026)
- [High Dynamic Range Video Reconstruction from Single-Exposure Raw Sequences](https://mlsanity.com/paper/2609.27274/) (2026)

## Abstract (arXiv)

As an important and practical way to obtain high dynamic range (HDR) video, HDR video reconstruction from sequences with alternating exposures is still less explored, mainly due to the lack of large-scale real-world datasets. Existing methods are mostly trained on synthetic datasets, which perform poorly in real scenes. In this work, to facilitate the development of real-world HDR video reconstruction, we present Real-HDRV, a large-scale real-world benchmark dataset for HDR video reconstruction, featuring various scenes, diverse motion patterns, and high-quality labels. Specifically, our dataset contains 500 LDRs-HDRs video pairs, comprising about 28,000 LDR frames and 4,000 HDR labels, covering daytime, nighttime, indoor, and outdoor scenes. To our best knowledge, our dataset is the largest real-world HDR video reconstruction dataset. Correspondingly, we propose an end-to-end network for HDR video reconstruction, where a novel two-stage strategy is designed to perform alignment sequentially. Specifically, the first stage performs global alignment with the adaptively estimated global offsets, reducing the difficulty of subsequent alignment. The second stage implicitly performs local alignment in a coarse-to-fine manner at the feature level using the adaptive separable convolution. Extensive experiments demonstrate that: (1) models trained on our dataset can achieve better performance on real scenes than those trained on synthetic datasets; (2) our method outperforms previous state-of-the-art methods. Our dataset is available at https://github.com/yungsyu99/Real-HDRV.