# How to cheat with metrics in single-image HDR reconstruction

arXiv: https://arxiv.org/abs/2108.08713 · HTML: https://mlsanity.com/paper/2108.08713/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 42 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 13 of 13)

- [HDR image reconstruction from a single exposure using deep CNNs](https://mlsanity.com/paper/1710.07480/) (2017)
- [Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline](https://mlsanity.com/paper/2004.01179/) (2020)
- [Attention-guided Network for Ghost-free High Dynamic Range Imaging](https://mlsanity.com/paper/1904.10293/) (2019)
- [Deep High Dynamic Range Imaging with Large Foreground Motions](https://mlsanity.com/paper/1711.08937/) (2017)
- [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018)
- [Single Image HDR Reconstruction Using a CNN with Masked Features and Perceptual Loss](https://mlsanity.com/paper/2005.07335/) (2020)
- [HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset](https://mlsanity.com/paper/2103.14943/) (2021)
- [Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications](https://mlsanity.com/paper/1904.11176/) (2019)
- [NTIRE 2021 Challenge on High Dynamic Range Imaging: Dataset, Methods and Results](https://mlsanity.com/paper/2106.01439/) (2021)
- [Learning High Dynamic Range from Outdoor Panoramas](https://mlsanity.com/paper/1703.10200/) (2017)
- [JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video](https://mlsanity.com/paper/1909.04391/) (2019)
- [FHDR: HDR Image Reconstruction from a Single LDR Image using Feedback Network](https://mlsanity.com/paper/1912.11463/) (2019)
- [Noise-Aware Merging of High Dynamic Range Image Stacks without Camera Calibration](https://mlsanity.com/paper/2009.07975/) (2020)

## Built on by (most cited first, 11 of 11)

- [High Dynamic Range and Super-Resolution from Raw Image Bursts](https://mlsanity.com/paper/2207.14671/) (2022)
- [Learning a Practical SDR-to-HDRTV Up-conversion using New Dataset and Degradation Models](https://mlsanity.com/paper/2303.13031/) (2023)
- [4K-Resolution Photo Exposure Correction at 125 FPS with ~8K Parameters](https://mlsanity.com/paper/2311.08759/) (2023)
- [LHDR: HDR Reconstruction for Legacy Content using a Lightweight DNN](https://mlsanity.com/paper/2211.11270/) (2022)
- [Perceptual Assessment and Optimization of HDR Image Rendering](https://mlsanity.com/paper/2310.12877/) (2023)
- [Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping](https://mlsanity.com/paper/2209.15165/) (2022)
- [A Perceptually Optimized and Self-Calibrated Tone Mapping Operator](https://mlsanity.com/paper/2206.09146/) (2022)
- [Joint Super-Resolution and Inverse Tone-Mapping: A Feature Decomposition Aggregation Network and A New Benchmark](https://mlsanity.com/paper/2207.03367/) (2022)
- [Single-Image HDR Reconstruction Assisted Ghost Suppression and Detail Preservation Network for Multi-Exposure HDR Imaging](https://mlsanity.com/paper/2403.04228/) (2024)
- [Redistributing the Precision and Content in 3D-LUT-based Inverse Tone-mapping for HDR/WCG Display](https://mlsanity.com/paper/2309.17160/) (2023)
- [Lucky High Dynamic Range Smartphone Imaging](https://mlsanity.com/paper/2604.19976/) (2026)

## Abstract (arXiv)

Single-image high dynamic range (SI-HDR) reconstruction has recently emerged as a problem well-suited for deep learning methods. Each successive technique demonstrates an improvement over existing methods by reporting higher image quality scores. This paper, however, highlights that such improvements in objective metrics do not necessarily translate to visually superior images. The first problem is the use of disparate evaluation conditions in terms of data and metric parameters, calling for a standardized protocol to make it possible to compare between papers. The second problem, which forms the main focus of this paper, is the inherent difficulty in evaluating SI-HDR reconstructions since certain aspects of the reconstruction problem dominate objective differences, thereby introducing a bias. Here, we reproduce a typical evaluation using existing as well as simulated SI-HDR methods to demonstrate how different aspects of the problem affect objective quality metrics. Surprisingly, we found that methods that do not even reconstruct HDR information can compete with state-of-the-art deep learning methods. We show how such results are not representative of the perceived quality and that SI-HDR reconstruction needs better evaluation protocols.