mlsanityData to 7 Oct 2026

Subhomogeneous Deep Equilibrium Models

arXiv 2403.00720 ↗First posted 1 Mar 2024cs.LG cs.NA math.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

Implicit-depth neural networks have grown as powerful alternatives to traditional networks in various applications in recent years. However, these models often lack guarantees of existence and uniqueness, raising stability, performance, and reproducibility issues. In this paper, we present a new analysis of the existence and uniqueness of fixed points for implicit-depth neural networks based on the concept of subhomogeneous operators and the nonlinear Perron-Frobenius theory. Compared to previous similar analyses, our theory allows for weaker assumptions on the parameter matrices, thus yielding a more flexible framework for well-defined implicit networks. We illustrate the performance of the resulting subhomogeneous networks on feedforward, convolutional, and graph neural network examples.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 1 Mar 2024.

Builds on

26 of 35 references.

References · 26

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2016Layer Normalizationcited by 5,579
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2018Predict then Propagate: Graph Neural Networks meet Personalized PageRankcited by 987
  5. 2018Universal Transformerscited by 613
  6. 2019Deep Equilibrium Modelscited by 603
  7. 2020Multiscale Deep Equilibrium Modelscited by 149
  8. 2021Deep Equilibrium Architectures for Inverse Problems in Imagingcited by 126
  9. 2019Implicit Deep Learningcited by 124
  10. 2020Monotone operator equilibrium networkscited by 100
  11. 2020Implicit Graph Neural Networkscited by 98
  12. 2022Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvaturecited by 96
14 more
  1. 2020Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependenciescited by 81
  2. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  3. 2022Deep Equilibrium Optical Flow Estimationcited by 63
  4. 2022On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networkscited by 63
  5. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 48
  6. 2022Deep Equilibrium Approaches to Diffusion Modelscited by 42
  7. 2018Recurrent Stacking of Layers for Compact Neural Machine Translation Modelscited by 38
  8. 2023Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Modelscited by 26
  9. 2021Robust Implicit Networks via Non-Euclidean Contractionscited by 19
  10. 2021Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certificationcited by 16
  11. 2023A Closer Look at the Adversarial Robustness of Deep Equilibrium Modelscited by 13
  12. 2021Fixed points of nonnegative neural networkscited by 5
  13. 2022Deep Equilibrium Models for Video Snapshot Compressive Imagingcited by 5
  14. 2018The Perron-Frobenius theorem for multi-homogeneous mappingscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Are We Measuring Oversmoothing in Graph Neural Networks Correctly?cited by 6
  2. 2026Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approachcited by 1
  3. 2026Certified Inference and Training for Deep Equilibrium Networks: A Continuation Framework with Polynomial Complexity Guaranteescited by 0