# Debiased Subjective Assessment of Real-World Image Enhancement

arXiv: https://arxiv.org/abs/2106.10080 · HTML: https://mlsanity.com/paper/2106.10080/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 87 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 30 of 37)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [DehazeNet: An End-to-End System for Single Image Haze Removal](https://mlsanity.com/paper/1601.07661/) (2016)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Learning a Single Convolutional Super-Resolution Network for Multiple Degradations](https://mlsanity.com/paper/1712.06116/) (2017)
- [Deep Back-Projection Networks For Super-Resolution](https://mlsanity.com/paper/1803.02735/) (2018)
- [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020)
- [Getting to Know Low-light Images with The Exclusively Dark Dataset](https://mlsanity.com/paper/1805.11227/) (2018)
- [Densely Connected Pyramid Dehazing Network](https://mlsanity.com/paper/1803.08396/) (2018)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Learning a No-Reference Quality Metric for Single-Image Super-Resolution](https://mlsanity.com/paper/1612.05890/) (2016)
- [Gated Fusion Network for Single Image Dehazing](https://mlsanity.com/paper/1804.00213/) (2018)
- [Domain Adaptation for Image Dehazing](https://mlsanity.com/paper/2005.04668/) (2020)
- [Gated Context Aggregation Network for Image Dehazing and Deraining](https://mlsanity.com/paper/1811.08747/) (2018)

## Built on by (most cited first, 7 of 7)

- [Locally Adaptive Structure and Texture Similarity for Image Quality Assessment](https://mlsanity.com/paper/2110.08521/) (2021)
- [Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition](https://mlsanity.com/paper/2404.08008/) (2024)
- [Image Quality Assessment: Integrating Model-Centric and Data-Centric Approaches](https://mlsanity.com/paper/2207.14769/) (2022)
- [A Perceptually Optimized and Self-Calibrated Tone Mapping Operator](https://mlsanity.com/paper/2206.09146/) (2022)
- [Quality-guided Skin Tone Enhancement for Portrait Photography](https://mlsanity.com/paper/2406.15848/) (2024)
- [HiLLIE: Human-in-the-Loop Training for Low-Light Image Enhancement](https://mlsanity.com/paper/2505.02134/) (2025)
- [Fine-Grained Anomaly Perception in Wild UGC-Enhanced Images: A Comprehensive Dataset and Difference-Fusion Framework](https://mlsanity.com/paper/2609.02529/) (2026)

## Abstract (arXiv)

In real-world image enhancement, it is often challenging (if not impossible) to acquire ground-truth data, preventing the adoption of distance metrics for objective quality assessment. As a result, one often resorts to subjective quality assessment, the most straightforward and reliable means of evaluating image enhancement. Conventional subjective testing requires manually pre-selecting a small set of visual examples, which may suffer from three sources of biases: 1) sampling bias due to the extremely sparse distribution of the selected samples in the image space; 2) algorithmic bias due to potential overfitting the selected samples; 3) subjective bias due to further potential cherry-picking test results. This eventually makes the field of real-world image enhancement more of an art than a science. Here we take steps towards debiasing conventional subjective assessment by automatically sampling a set of adaptive and diverse images for subsequent testing. This is achieved by casting sample selection into a joint maximization of the discrepancy between the enhancers and the diversity among the selected input images. Careful visual inspection on the resulting enhanced images provides a debiased ranking of the enhancement algorithms. We demonstrate our subjective assessment method using three popular and practically demanding image enhancement tasks: dehazing, super-resolution, and low-light enhancement.