# Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks

arXiv: https://arxiv.org/abs/2310.06958 · HTML: https://mlsanity.com/paper/2310.06958/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 94 references that are in the map.
- First posted on arXiv 10 Oct 2023.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [Neural Optimal Transport](https://mlsanity.com/paper/2201.12220/) (2022)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [Collaborative Learning for Faster StyleGAN Embedding](https://mlsanity.com/paper/2007.01758/) (2020)
- [Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment](https://mlsanity.com/paper/2008.03889/) (2020)
- [E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles](https://mlsanity.com/paper/1906.03973/) (2019)
- [NTIRE 2022 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2206.11695/) (2022)
- [Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop](https://mlsanity.com/paper/2210.00933/) (2022)
- [Video compression dataset and benchmark of learning-based video-quality metrics](https://mlsanity.com/paper/2211.12109/) (2022)
- [NTIRE 2021 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2105.03072/) (2021)

## Built on by (most cited first, 14 of 14)

- [One Surrogate to Fool Them All: Universal, Transferable, and Targeted Adversarial Attacks with CLIP](https://mlsanity.com/paper/2505.19840/) (2025)
- [Cross-Modal Transferable Image-to-Video Attack on Video Quality Metrics](https://mlsanity.com/paper/2501.08415/) (2025)
- [Defense Against Adversarial Attacks on No-Reference Image Quality Models with Gradient Norm Regularization](https://mlsanity.com/paper/2403.11397/) (2024)
- [Adversarial purification for no-reference image-quality metrics: applicability study and new methods](https://mlsanity.com/paper/2404.06957/) (2024)
- [Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics](https://mlsanity.com/paper/2408.01541/) (2024)
- [Ti-Patch: Tiled Physical Adversarial Patch for no-reference video quality metrics](https://mlsanity.com/paper/2404.09961/) (2024)
- [Can No-Reference Quality-Assessment Methods Serve as Perceptual Losses for Super-Resolution?](https://mlsanity.com/paper/2405.20392/) (2024)
- [Beyond Score Changes: Adversarial Attack on No-Reference Image Quality Assessment from Two Perspectives](https://mlsanity.com/paper/2404.13277/) (2024)
- [Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunities](https://mlsanity.com/paper/2410.08534/) (2024)
- [Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods](https://mlsanity.com/paper/2411.11795/) (2024)
- [Neural Atlas Graphs for Dynamic Scene Decomposition and Editing](https://mlsanity.com/paper/2509.16336/) (2025)
- [NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis](https://mlsanity.com/paper/2506.19051/) (2025)
- [BiRQA: Bidirectional Robust Quality Assessment for Images](https://mlsanity.com/paper/2602.20351/) (2026)
- [Saving Foundation Flow-Matching Priors for Inverse Problems](https://mlsanity.com/paper/2511.16520/) (2025)

## Abstract (arXiv)

Nowadays, neural-network-based image- and video-quality metrics perform better than traditional methods. However, they also became more vulnerable to adversarial attacks that increase metrics' scores without improving visual quality. The existing benchmarks of quality metrics compare their performance in terms of correlation with subjective quality and calculation time. Nonetheless, the adversarial robustness of image-quality metrics is also an area worth researching. This paper analyses modern metrics' robustness to different adversarial attacks. We adapted adversarial attacks from computer vision tasks and compared attacks' efficiency against 15 no-reference image- and video-quality metrics. Some metrics showed high resistance to adversarial attacks, which makes their usage in benchmarks safer than vulnerable metrics. The benchmark accepts submissions of new metrics for researchers who want to make their metrics more robust to attacks or to find such metrics for their needs. The latest results can be found online: https://videoprocessing.ai/benchmarks/metrics-robustness.html.