mlsanityData to 7 Oct 2026

Lipschitz Bounded Equilibrium Networks

arXiv 2010.01732 ↗First posted 5 Oct 2020cs.LG cs.CV cs.SYcited by 49

Not on the map. Why?

Abstract (arXiv)

This paper introduces new parameterizations of equilibrium neural networks, i.e. networks defined by implicit equations. This model class includes standard multilayer and residual networks as special cases. The new parameterization admits a Lipschitz bound during training via unconstrained optimization: no projections or barrier functions are required. Lipschitz bounds are a common proxy for robustness and appear in many generalization bounds. Furthermore, compared to previous works we show well-posedness (existence of solutions) under less restrictive conditions on the network weights and more natural assumptions on the activation functions: that they are monotone and slope restricted. These results are proved by establishing novel connections with convex optimization, operator splitting on non-Euclidean spaces, and contracting neural ODEs. In image classification experiments we show that the Lipschitz bounds are very accurate and improve robustness to adversarial attacks.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 14; 2024: 9; 2025: 5; 2026: 5.

49 papers.

Sort:

By area:

  1. 2021On Training Implicit Modelscited by 73
  2. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  3. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 60
  4. 2023A Unified Algebraic Perspective on Lipschitz Neural Networkscited by 58
  5. 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 49
  6. 2021Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustnesscited by 43
  7. 2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 33
  8. 2023Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Modelscited by 26
  9. 2023Some Fundamental Aspects about Lipschitz Continuity of Neural Networkscited by 25
  10. 2023TorchDEQ: A Library for Deep Equilibrium Modelscited by 21
  11. 2021Robust Implicit Networks via Non-Euclidean Contractionscited by 19
  12. 2023Deep Equilibrium Based Neural Operators for Steady-State PDEscited by 19
37 more
  1. 2021Joint inference and input optimization in equilibrium networkscited by 15
  2. 2023A Closer Look at the Adversarial Robustness of Deep Equilibrium Modelscited by 13
  3. 2022Global Convergence of Over-parameterized Deep Equilibrium Modelscited by 10
  4. 2023Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramiancited by 9
  5. 2024Monotone, Bi-Lipschitz, and Polyak-Lojasiewicz Networkscited by 9
  6. 2021Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approachcited by 8
  7. 2022Deep equilibrium networks are sensitive to initialization statisticscited by 8
  8. 2024Lipschitz constant estimation for general neural network architectures using control toolscited by 8
  9. 2021RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networkscited by 7
  10. 2024Positive concave deep equilibrium modelscited by 7
  11. 2024Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activationscited by 6
  12. 2021Optimization Induced Equilibrium Networkscited by 5
  13. 2023Convolutional Neural Networks as 2-D systemscited by 5
  14. 2022Deep equilibrium models as estimators for continuous latent variablescited by 4
  15. 2024On Robust Reinforcement Learning with Lipschitz-Bounded Policy Networkscited by 4
  16. 2024State space representations of the Roesser type for convolutional layerscited by 3
  17. 2026Quantization Robustness of Monotone Operator Equilibrium Networkscited by 3
  18. 2022Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networkscited by 2
  19. 2023Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamicscited by 2
  20. 2023Accelerating Hopfield Network Dynamics: Beyond Synchronous Updates and Forward Eulercited by 2
  21. 2024A Complete Set of Quadratic Constraints for Repeated ReLU and Generalizationscited by 2
  22. 2024LipKernel: Lipschitz-Bounded Convolutional Neural Networks via Dissipative Layerscited by 2
  23. 2024Certified Robustness for Deep Equilibrium Models via Serialized Random Smoothingcited by 2
  24. 2025Improved Scalable Lipschitz Bounds for Deep Neural Networkscited by 2
  25. 2025R2DN: Scalable Parameterization of Contracting and Lipschitz Recurrent Deep Networkscited by 2
  26. 2021Abstract Interpretation of Fixpoint Iterators with Applications to Neural Networkscited by 1
  27. 2023Learning Stable and Robust Linear Parameter-Varying State-Space Modelscited by 1
  28. 2025Robustly Invertible Nonlinear Dynamics and the BiLipREN: Contracting Neural Models with Contracting Inversescited by 1
  29. 2025Circuit realization and hardware linearization of monotone operator equilibrium networkscited by 1
  30. 2025Expressive Power of Implicit Models: Rich Equilibria and Test-Time Scalingcited by 1
  31. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1
  32. 2022Representation Recycling for Streaming Video Analysiscited by 0
  33. 2022Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approachcited by 0
  34. 2023Monotone deep Boltzmann machinescited by 0
  35. 2026A Nonlinear Separation Principle via Contraction Theory: Applications to Neural Networks, Control, and Learningcited by 0
  36. 2026Bifurcation Models: Learning Set-Valued Solution Maps with Weight-Tied Dynamicscited by 0
  37. 2026Robustly Invertible Nonlinear Dynamics and the BiLipREN: From Inversion-Based Control to Generative Trajectory Modellingcited by 0