Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics
Not on the map. Why?
Abstract (arXiv)
Deep equilibrium (DEQ) models replace the multiple-layer stacking of conventional deep networks with a fixed-point iteration of a single-layer transformation. Having been demonstrated to be competitive in a variety of real-world scenarios, the adversarial robustness of general DEQs becomes increasingly crucial for their reliable deployment. Existing works improve the robustness of general DEQ models with the widely-used adversarial training (AT) framework, but they fail to exploit the structural uniquenesses of DEQ models. To this end, we interpret DEQs through the lens of neural dynamics and find that AT under-regulates intermediate states. Besides, the intermediate states typically provide predictions with a high prediction entropy. Informed by the correlation between the entropy of dynamical systems and their stability properties, we propose reducing prediction entropy by progressively updating inputs along the neural dynamics. During AT, we also utilize random intermediate states to compute the loss function. Our methods regulate the neural dynamics of DEQ models in this manner. Extensive experiments demonstrate that our methods substantially increase the robustness of DEQ models and even outperform the strong deep network baselines.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 2 Jun 2023.
Builds on
42 of 60 references.
References · 42
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2016Wide Residual Networkscited by 3,057
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
- 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
- 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
- 2019Deep Equilibrium Modelscited by 603
30 more
- 2020Overfitting in adversarially robust deep learningcited by 504
- 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
- 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
- 2020Bag of Tricks for Adversarial Trainingcited by 199
- 2021Training Graph Neural Networks with 1000 Layerscited by 169
- 2020Multiscale Deep Equilibrium Modelscited by 149
- 2021Adversarial purification with Score-based generative modelscited by 132
- 2022Robustness and Accuracy Could Be Reconcilable by (Proper) Definitioncited by 109
- 2020Monotone operator equilibrium networkscited by 100
- 2021Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attackscited by 91
- 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
- 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
- 2019On Robustness of Neural Ordinary Differential Equationscited by 83
- 2017Maximum Principle Based Algorithms for Deep Learningcited by 77
- 2021On Training Implicit Modelscited by 73
- 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
- 2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
- 2022Deep Equilibrium Optical Flow Estimationcited by 63
- 2019Mixup Inference: Better Exploiting Mixup to Defend Adversarial Attackscited by 54
- 2020Lipschitz Bounded Equilibrium Networkscited by 49
- 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47
- 2021Online Adversarial Purification based on Self-Supervisioncited by 45
- 2022Deep Equilibrium Approaches to Diffusion Modelscited by 42
- 2021Attacking Adversarial Attacks as A Defensecited by 31
- 2021Implicit Normalizing Flowscited by 28
- 2021Combating Adversaries with Anti-Adversariescited by 27
- 2021Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certificationcited by 16
- 2021Joint inference and input optimization in equilibrium networkscited by 15
- 2021Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approachcited by 8
- 2020Interpolation between Residual and Non-Residual Networkscited by 6
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.