# Robustness as Architecture: Designing IQA Models to Withstand Adversarial Perturbations

arXiv: https://arxiv.org/abs/2506.04951 · HTML: https://mlsanity.com/paper/2506.04951/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 29 of 29)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [A guide to convolution arithmetic for deep learning](https://mlsanity.com/paper/1603.07285/) (2016)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [Orthogonalizing Convolutional Layers with the Cayley Transform](https://mlsanity.com/paper/2104.07167/) (2021)
- [Smooth Adversarial Training](https://mlsanity.com/paper/2006.14536/) (2020)
- [Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment](https://mlsanity.com/paper/2008.03889/) (2020)
- [Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks](https://mlsanity.com/paper/2208.03160/) (2022)
- [Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop](https://mlsanity.com/paper/2210.00933/) (2022)
- [On the Effect of Pruning on Adversarial Robustness](https://mlsanity.com/paper/2108.04890/) (2021)
- [Universal Perturbation Attack on Differentiable No-Reference Image- and Video-Quality Metrics](https://mlsanity.com/paper/2211.00366/) (2022)
- [Pruning Adversarially Robust Neural Networks without Adversarial Examples](https://mlsanity.com/paper/2210.04311/) (2022)
- [An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures](https://mlsanity.com/paper/2501.07930/) (2025)
- [Fast Adversarial CNN-based Perturbation Attack on No-Reference Image- and Video-Quality Metrics](https://mlsanity.com/paper/2305.15544/) (2023)
- [Defense Against Adversarial Attacks on No-Reference Image Quality Models with Gradient Norm Regularization](https://mlsanity.com/paper/2403.11397/) (2024)
- [Adversarial purification for no-reference image-quality metrics: applicability study and new methods](https://mlsanity.com/paper/2404.06957/) (2024)
- [Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics](https://mlsanity.com/paper/2408.01541/) (2024)
- [Pruning Deep Neural Networks using Partial Least Squares](https://mlsanity.com/paper/1810.07610/) (2018)
- [A Non-monotonic Smooth Activation Function](https://mlsanity.com/paper/2310.10126/) (2023)

## Abstract (arXiv)

Image Quality Assessment (IQA) models are increasingly relied upon to evaluate image quality in real-world systems -- from compression and enhancement to generation and streaming. Yet their adoption brings a fundamental risk: these models are inherently unstable. Adversarial manipulations can easily fool them, inflating scores and undermining trust. Traditionally, such vulnerabilities are addressed through data-driven defenses -- adversarial retraining, regularization, or input purification. But what if this is the wrong lens? What if robustness in perceptual models is not something to learn but something to design? In this work, we propose a provocative idea: robustness as an architectural prior. Rather than training models to resist perturbations, we reshape their internal structure to suppress sensitivity from the ground up. We achieve this by enforcing orthogonal information flow, constraining the network to norm-preserving operations -- and further stabilizing the system through pruning and fine-tuning. The result is a robust IQA architecture that withstands adversarial attacks without requiring adversarial training or significant changes to the original model. This approach suggests a shift in perspective: from optimizing robustness through data to engineering it through design.