# Towards True Detail Restoration for Super-Resolution: A Benchmark and a Quality Metric

arXiv: https://arxiv.org/abs/2203.08923 · HTML: https://mlsanity.com/paper/2203.08923/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 29 of 29)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [VRT: A Video Restoration Transformer](https://mlsanity.com/paper/2201.12288/) (2022)
- [PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restoration](https://mlsanity.com/paper/2007.12142/) (2020)
- [Recurrent Back-Projection Network for Video Super-Resolution](https://mlsanity.com/paper/1903.10128/) (2019)
- [Video Super-Resolution with Recurrent Structure-Detail Network](https://mlsanity.com/paper/2008.00455/) (2020)
- [Handheld Multi-Frame Super-Resolution](https://mlsanity.com/paper/1905.03277/) (2019)
- [Video Super-resolution with Temporal Group Attention](https://mlsanity.com/paper/2007.10595/) (2020)
- [Temporal Modulation Network for Controllable Space-Time Video Super-Resolution](https://mlsanity.com/paper/2104.10642/) (2021)
- [Revisiting Temporal Modeling for Video Super-resolution](https://mlsanity.com/paper/2008.05765/) (2020)
- [Deep Blind Video Super-resolution](https://mlsanity.com/paper/2003.04716/) (2020)
- [Deep Video Super-Resolution using HR Optical Flow Estimation](https://mlsanity.com/paper/2001.02129/) (2020)
- [Deformable 3D Convolution for Video Super-Resolution](https://mlsanity.com/paper/2004.02803/) (2020)
- [Learning for Video Super-Resolution through HR Optical Flow Estimation](https://mlsanity.com/paper/1809.08573/) (2018)
- [NTIRE 2021 Challenge on Video Super-Resolution](https://mlsanity.com/paper/2104.14852/) (2021)
- [Image Super-Resolution Quality Assessment: Structural Fidelity Versus Statistical Naturalness](https://mlsanity.com/paper/2105.07139/) (2021)
- [ERQA: Edge-Restoration Quality Assessment for Video Super-Resolution](https://mlsanity.com/paper/2110.09992/) (2021)
- [Edge and Identity Preserving Network for Face Super-Resolution](https://mlsanity.com/paper/2008.11977/) (2020)
- [DynaVSR: Dynamic Adaptive Blind Video Super-Resolution](https://mlsanity.com/paper/2011.04482/) (2020)
- [iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks](https://mlsanity.com/paper/2006.11161/) (2020)

## Built on by (most cited first, 5 of 5)

- [Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks](https://mlsanity.com/paper/2310.06958/) (2023)
- [Combining Contrastive and Supervised Learning for Video Super-Resolution Detection](https://mlsanity.com/paper/2205.10406/) (2022)
- [Creating Realistic Anterior Segment Optical Coherence Tomography Images using Generative Adversarial Networks](https://mlsanity.com/paper/2306.14058/) (2023)
- [A Survey on Super Resolution for video Enhancement Using GAN](https://mlsanity.com/paper/2312.16471/) (2023)
- [MSIQ: Moment-based Scale-Invariant Quality Measure for Single Image Super-Resolution](https://mlsanity.com/paper/2605.17588/) (2026)

## Abstract (arXiv)

Super-resolution (SR) has become a widely researched topic in recent years. SR methods can improve overall image and video quality and create new possibilities for further content analysis. But the SR mainstream focuses primarily on increasing the naturalness of the resulting image despite potentially losing context accuracy. Such methods may produce an incorrect digit, character, face, or other structural object even though they otherwise yield good visual quality. Incorrect detail restoration can cause errors when detecting and identifying objects both manually and automatically. To analyze the detail-restoration capabilities of image and video SR models, we developed a benchmark based on our own video dataset, which contains complex patterns that SR models generally fail to correctly restore. We assessed 32 recent SR models using our benchmark and compared their ability to preserve scene context. We also conducted a crowd-sourced comparison of restored details and developed an objective assessment metric that outperforms other quality metrics by correlation with subjective scores for this task. In conclusion, we provide a deep analysis of benchmark results that yields insights for future SR-based work.