# Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics

arXiv: https://arxiv.org/abs/2408.01541 · HTML: https://mlsanity.com/paper/2408.01541/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 75 references that are in the map.
- First posted on arXiv 2 Aug 2024.

## Builds on (most cited first, 30 of 47)

- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Adversarial Attacks on Neural Network Policies](https://mlsanity.com/paper/1702.02284/) (2017)
- [SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution](https://mlsanity.com/paper/2311.16518/) (2023)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2306.04717/) (2023)
- [(Certified!!) Adversarial Robustness for Free!](https://mlsanity.com/paper/2206.10550/) (2022)
- [Unrestricted Adversarial Examples via Semantic Manipulation](https://mlsanity.com/paper/1904.06347/) (2019)
- [Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models](https://mlsanity.com/paper/2303.01870/) (2023)
- [Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity](https://mlsanity.com/paper/2203.05151/) (2022)
- [Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks](https://mlsanity.com/paper/2006.12834/) (2020)
- [Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment](https://mlsanity.com/paper/2008.03889/) (2020)
- [Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimization](https://mlsanity.com/paper/1905.06635/) (2019)

## Built on by (most cited first, 5 of 5)

- [Augmenting Perceptual Super-Resolution via Image Quality Predictors](https://mlsanity.com/paper/2504.18524/) (2025)
- [Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunities](https://mlsanity.com/paper/2410.08534/) (2024)
- [BiRQA: Bidirectional Robust Quality Assessment for Images](https://mlsanity.com/paper/2602.20351/) (2026)
- [Stochastic BIQA: Median Randomized Smoothing for Certified Blind Image Quality Assessment](https://mlsanity.com/paper/2411.12575/) (2024)
- [Robustness as Architecture: Designing IQA Models to Withstand Adversarial Perturbations](https://mlsanity.com/paper/2506.04951/) (2025)

## Abstract (arXiv)

In the field of Image Quality Assessment (IQA), the adversarial robustness of the metrics poses a critical concern. This paper presents a comprehensive benchmarking study of various defense mechanisms in response to the rise in adversarial attacks on IQA. We systematically evaluate 25 defense strategies, including adversarial purification, adversarial training, and certified robustness methods. We applied 14 adversarial attack algorithms of various types in both non-adaptive and adaptive settings and tested these defenses against them. We analyze the differences between defenses and their applicability to IQA tasks, considering that they should preserve IQA scores and image quality. The proposed benchmark aims to guide future developments and accepts submissions of new methods, with the latest results available online: https://videoprocessing.ai/benchmarks/iqa-defenses.html.