mlsanityData to 14 Oct 2026

A Closer Look at the Adversarial Robustness of Deep Equilibrium Models

arXiv 2306.01429 ↗First posted 2 Jun 2023cs.LG stat.MLcited by 13

Not on the map. Why?

Abstract (arXiv)

Deep equilibrium models (DEQs) refrain from the traditional layer-stacking paradigm and turn to find the fixed point of a single layer. DEQs have achieved promising performance on different applications with featured memory efficiency. At the same time, the adversarial vulnerability of DEQs raises concerns. Several works propose to certify robustness for monotone DEQs. However, limited efforts are devoted to studying empirical robustness for general DEQs. To this end, we observe that an adversarially trained DEQ requires more forward steps to arrive at the equilibrium state, or even violates its fixed-point structure. Besides, the forward and backward tracks of DEQs are misaligned due to the black-box solvers. These facts cause gradient obfuscation when applying the ready-made attacks to evaluate or adversarially train DEQs. Given this, we develop approaches to estimate the intermediate gradients of DEQs and integrate them into the attacking pipelines. Our approaches facilitate fully white-box evaluations and lead to effective adversarial defense for DEQs. Extensive experiments on CIFAR-10 validate the adversarial robustness of DEQs competitive with deep networks of similar sizes.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 36 of its 46 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

36 of 46 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2018Neural Ordinary Differential Equationscited by 3,779
  7. 2016Wide Residual Networkscited by 3,070
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2016Adversarial Machine Learning at Scalecited by 936
24 more
  1. 2018Robustness May Be at Odds with Accuracycited by 880
  2. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  3. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  4. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  5. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  6. 2019Deep Equilibrium Modelscited by 615
  7. 2020Overfitting in adversarially robust deep learningcited by 506
  8. 2019On Evaluating Adversarial Robustnesscited by 447
  9. 2020Bag of Tricks for Adversarial Trainingcited by 200
  10. 2021Training Graph Neural Networks with 1000 Layerscited by 169
  11. 2020Multiscale Deep Equilibrium Modelscited by 151
  12. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  13. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  14. 2020Monotone operator equilibrium networkscited by 100
  15. 2021On Training Implicit Modelscited by 75
  16. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 72
  17. 2020Lipschitz Bounded Equilibrium Networkscited by 49
  18. 2021Implicit Normalizing Flowscited by 28
  19. 2021SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit modelscited by 26
  20. 2021Towards Robust Neural Networks via Close-loop Controlcited by 23
  21. 2020Adversarial Robustness of Stabilized NeuralODEs Might be from Obfuscated Gradientscited by 22
  22. 2021Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certificationcited by 16
  23. 2021Joint inference and input optimization in equilibrium networkscited by 15
  24. 2021Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approachcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 3; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023TorchDEQ: A Library for Deep Equilibrium Modelscited by 21
  2. 2023Deep Equilibrium Diffusion Restoration with Parallel Samplingcited by 17
  3. 2024Texture Re-scalable Universal Adversarial Perturbationcited by 9
  4. 2024Adversarially Robust Out-of-Distribution Detection Using Lyapunov-Stabilized Embeddingscited by 6
  5. 2024Deep Implicit Optimization enables Robust Learnable Features for Deformable Image Registrationcited by 5
  6. 2024Subhomogeneous Deep Equilibrium Modelscited by 3
  7. 2024Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspectivecited by 3
  8. 2024Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothingcited by 2
  9. 2025Adversarial Attacks in Weight-Space Classifierscited by 1
  10. 2025Stochastic Sample Approximations of (Local) Moduli of Continuitycited by 1
  11. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1
  12. 2024How Robust Are Energy-Based Models Trained With Equilibrium Propagation?cited by 0
1 more
  1. 2025Training-Free Adversarial Robustness in Computational MRIcited by 0