# Subhomogeneous Deep Equilibrium Models

arXiv: https://arxiv.org/abs/2403.00720 · HTML: https://mlsanity.com/paper/2403.00720/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 35 references that are in the map.
- First posted on arXiv 1 Mar 2024.

## Builds on (most cited first, 26 of 26)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Predict then Propagate: Graph Neural Networks meet Personalized PageRank](https://mlsanity.com/paper/1810.05997/) (2018)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [Implicit Graph Neural Networks](https://mlsanity.com/paper/2009.06211/) (2020)
- [Revisiting Over-smoothing and Over-squashing Using Ollivier-Ricci Curvature](https://mlsanity.com/paper/2211.15779/) (2022)
- [Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies](https://mlsanity.com/paper/2010.00951/) (2020)
- [Generalizable Adversarial Training via Spectral Normalization](https://mlsanity.com/paper/1811.07457/) (2018)
- [Deep Equilibrium Optical Flow Estimation](https://mlsanity.com/paper/2204.08442/) (2022)
- [On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks](https://mlsanity.com/paper/2212.02374/) (2022)
- [Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspective](https://mlsanity.com/paper/2210.01787/) (2022)
- [Deep Equilibrium Approaches to Diffusion Models](https://mlsanity.com/paper/2210.12867/) (2022)
- [Recurrent Stacking of Layers for Compact Neural Machine Translation Models](https://mlsanity.com/paper/1807.05353/) (2018)
- [Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models](https://mlsanity.com/paper/2304.00600/) (2023)
- [Robust Implicit Networks via Non-Euclidean Contractions](https://mlsanity.com/paper/2106.03194/) (2021)
- [Semialgebraic Representation of Monotone Deep Equilibrium Models and Applications to Certification](https://mlsanity.com/paper/2106.01453/) (2021)
- [A Closer Look at the Adversarial Robustness of Deep Equilibrium Models](https://mlsanity.com/paper/2306.01429/) (2023)
- [Fixed points of nonnegative neural networks](https://mlsanity.com/paper/2106.16239/) (2021)
- [Deep Equilibrium Models for Video Snapshot Compressive Imaging](https://mlsanity.com/paper/2201.06931/) (2022)
- [The Perron-Frobenius theorem for multi-homogeneous mappings](https://mlsanity.com/paper/1801.05034/) (2018)

## Built on by (most cited first, 3 of 3)

- [Are We Measuring Oversmoothing in Graph Neural Networks Correctly?](https://mlsanity.com/paper/2502.04591/) (2025)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [Certified Inference and Training for Deep Equilibrium Networks: A Continuation Framework with Polynomial Complexity Guarantees](https://mlsanity.com/paper/2609.16485/) (2026)

## Abstract (arXiv)

Implicit-depth neural networks have grown as powerful alternatives to traditional networks in various applications in recent years. However, these models often lack guarantees of existence and uniqueness, raising stability, performance, and reproducibility issues. In this paper, we present a new analysis of the existence and uniqueness of fixed points for implicit-depth neural networks based on the concept of subhomogeneous operators and the nonlinear Perron-Frobenius theory. Compared to previous similar analyses, our theory allows for weaker assumptions on the parameter matrices, thus yielding a more flexible framework for well-defined implicit networks. We illustrate the performance of the resulting subhomogeneous networks on feedforward, convolutional, and graph neural network examples.