# Positive concave deep equilibrium models

arXiv: https://arxiv.org/abs/2402.04029 · HTML: https://mlsanity.com/paper/2402.04029/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 53 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 26 of 26)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Multiscale Deep Equilibrium Models](https://mlsanity.com/paper/2006.08656/) (2020)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [On Training Implicit Models](https://mlsanity.com/paper/2111.05177/) (2021)
- [Deep Equilibrium Optical Flow Estimation](https://mlsanity.com/paper/2204.08442/) (2022)
- [One-Step Diffusion Distillation via Deep Equilibrium Models](https://mlsanity.com/paper/2401.08639/) (2023)
- [Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators](https://mlsanity.com/paper/1903.01014/) (2019)
- [Lipschitz Bounded Equilibrium Networks](https://mlsanity.com/paper/2010.01732/) (2020)
- [Deep Equilibrium Approaches to Diffusion Models](https://mlsanity.com/paper/2210.12867/) (2022)
- [Deep Equilibrium Learning of Explicit Regularizers for Imaging Inverse Problems](https://mlsanity.com/paper/2303.05386/) (2023)
- [Deep Equilibrium Object Detection](https://mlsanity.com/paper/2308.09564/) (2023)
- [Connections between spectral properties of asymptotic mappings and solutions to wireless network problems](https://mlsanity.com/paper/1903.09793/) (2019)
- [Fixed points of nonnegative neural networks](https://mlsanity.com/paper/2106.16239/) (2021)
- [Music Source Separation with Deep Equilibrium Models](https://mlsanity.com/paper/2110.06494/) (2021)
- [Elementary Properties of Positive Concave Mappings with Applications to Network Planning and Optimization](https://mlsanity.com/paper/1505.03006/) (2015)
- [Connections between Deep Equilibrium and Sparse Representation Models with Application to Hyperspectral Image Denoising](https://mlsanity.com/paper/2203.15901/) (2022)
- [Deep equilibrium models as estimators for continuous latent variables](https://mlsanity.com/paper/2211.05943/) (2022)
- [Deep Equilibrium Models Meet Federated Learning](https://mlsanity.com/paper/2305.18646/) (2023)
- [The fixed point iteration of positive concave mappings converges geometrically if a fixed point exists: implications to wireless systems](https://mlsanity.com/paper/2110.11055/) (2021)
- [A unified approach to nonlinear Perron-Frobenius theory](https://mlsanity.com/paper/2111.01219/) (2021)
- [Two-timescale joint power control and beamforming design with applications to cell-free massive MIMO](https://mlsanity.com/paper/2312.02080/) (2023)

## Built on by (most cited first, 7 of 7)

- [Generative Neural Operators of Log-Complexity Can Simultaneously Solve Infinitely Many Convex Programs](https://mlsanity.com/paper/2508.14995/) (2025)
- [Reversible Deep Equilibrium Models](https://mlsanity.com/paper/2509.12917/) (2025)
- [DDEQs: Distributional Deep Equilibrium Models through Wasserstein Gradient Flows](https://mlsanity.com/paper/2503.01140/) (2025)
- [Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium](https://mlsanity.com/paper/2511.21882/) (2025)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [On The Finetuning of MLIPs Through the Lens of Iterated Maps With BPTT](https://mlsanity.com/paper/2512.01067/) (2025)
- [Certified Inference and Training for Deep Equilibrium Networks: A Continuation Framework with Polynomial Complexity Guarantees](https://mlsanity.com/paper/2609.16485/) (2026)

## Abstract (arXiv)

Deep equilibrium (DEQ) models are widely recognized as a memory efficient alternative to standard neural networks, achieving state-of-the-art performance in language modeling and computer vision tasks. These models solve a fixed point equation instead of explicitly computing the output, which sets them apart from standard neural networks. However, existing DEQ models often lack formal guarantees of the existence and uniqueness of the fixed point, and the convergence of the numerical scheme used for computing the fixed point is not formally established. As a result, DEQ models are potentially unstable in practice. To address these drawbacks, we introduce a novel class of DEQ models called positive concave deep equilibrium (pcDEQ) models. Our approach, which is based on nonlinear Perron-Frobenius theory, enforces nonnegative weights and activation functions that are concave on the positive orthant. By imposing these constraints, we can easily ensure the existence and uniqueness of the fixed point without relying on additional complex assumptions commonly found in the DEQ literature, such as those based on monotone operator theory in convex analysis. Furthermore, the fixed point can be computed with the standard fixed point algorithm, and we provide theoretical guarantees of its geometric convergence, which, in particular, simplifies the training process. Experiments demonstrate the competitiveness of our pcDEQ models against other implicit models.