Robust Feature Inference: A Test-time Defense Strategy using Spectral Projections
Not on the map. Why?
Abstract (arXiv)
Test-time defenses are used to improve the robustness of deep neural networks to adversarial examples during inference. However, existing methods either require an additional trained classifier to detect and correct the adversarial samples, or perform additional complex optimization on the model parameters or the input to adapt to the adversarial samples at test-time, resulting in a significant increase in the inference time compared to the base model. In this work, we propose a novel test-time defense strategy called Robust Feature Inference (RFI) that is easy to integrate with any existing (robust) training procedure without additional test-time computation. Based on the notion of robustness of features that we present, the key idea is to project the trained models to the most robust feature space, thereby reducing the vulnerability to adversarial attacks in non-robust directions. We theoretically characterize the subspace of the eigenspectrum of the feature covariance that is the most robust for a generalized additive model. Our extensive experiments on CIFAR-10, CIFAR-100, tiny ImageNet and ImageNet datasets for several robustness benchmarks, including the state-of-the-art methods in RobustBench show that RFI improves robustness across adaptive and transfer attacks consistently. We also compare RFI with adaptive test-time defenses to demonstrate the effectiveness of our proposed approach.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 52 of its 62 references that are in the map. First posted on arXiv 21 Jul 2023.
Builds on
52 of 62 references.
References · 52
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
- 2013Intriguing properties of neural networkscited by 5,114
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
38 more
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
- 2018Robustness May Be at Odds with Accuracycited by 873
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
- 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
- 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
- 2017Countering Adversarial Images using Input Transformationscited by 517
- 2020Overfitting in adversarially robust deep learningcited by 504
- 2022Diffusion Models for Adversarial Purificationcited by 469
- 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
- 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 333
- 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
- 2021Improving Robustness using Generated Datacited by 233
- 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
- 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
- 2017Towards Robust Neural Networks via Random Self-ensemblecited by 158
- 2019Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivationcited by 146
- 2018Are adversarial examples inevitable?cited by 113
- 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
- 2015Global Convergence of ADMM in Nonconvex Nonsmooth Optimizationcited by 98
- 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91
- 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
- 2019Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracycited by 49
- 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
- 2021Online Adversarial Purification based on Self-Supervisioncited by 45
- 2022Efficient and Effective Augmentation Strategy for Adversarial Trainingcited by 43
- 2021Attacking Adversarial Attacks as A Defensecited by 31
- 2021Combating Adversaries with Anti-Adversariescited by 27
- 2020Improving Calibration through the Relationship with Adversarial Robustnesscited by 26
- 2022Robust Models are less Over-Confidentcited by 25
- 2021Towards Robust Neural Networks via Close-loop Controlcited by 23
- 2020Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradientscited by 22
- 2022What Can the Neural Tangent Kernel Tell Us About Adversarial Robustness?cited by 21
- 2021Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attackscited by 19
- 2021AID-Purifier: A Light Auxiliary Network for Boosting Adversarial Defensecited by 10
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.