# Robust Implicit Networks via Non-Euclidean Contractions

arXiv: https://arxiv.org/abs/2106.03194 · HTML: https://mlsanity.com/paper/2106.03194/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 6 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [Provable defenses against adversarial examples via the convex outer adversarial polytope](https://mlsanity.com/paper/1711.00851/) (2017)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Differentiable Convex Optimization Layers](https://mlsanity.com/paper/1910.12430/) (2019)
- [Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks](https://mlsanity.com/paper/1906.04893/) (2019)
- [Differentiable MPC for End-to-end Planning and Control](https://mlsanity.com/paper/1810.13400/) (2018)
- [Implicit Deep Learning](https://mlsanity.com/paper/1908.06315/) (2019)
- [Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming](https://mlsanity.com/paper/1903.01287/) (2019)
- [Monotone operator equilibrium networks](https://mlsanity.com/paper/2006.08591/) (2020)
- [Training robust neural networks using Lipschitz bounds](https://mlsanity.com/paper/2005.02929/) (2020)
- [Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators](https://mlsanity.com/paper/1903.01014/) (2019)
- [Lipschitz Bounded Equilibrium Networks](https://mlsanity.com/paper/2010.01732/) (2020)
- [On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers](https://mlsanity.com/paper/2102.07346/) (2021)
- [On the Limitation of Convolutional Neural Networks in Recognizing Negative Images](https://mlsanity.com/paper/1703.06857/) (2017)
- [Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing](https://mlsanity.com/paper/2006.03712/) (2020)
- [Fixed Point Strategies in Data Science](https://mlsanity.com/paper/2008.02260/) (2020)
- [Non-Euclidean Contraction Theory for Robust Nonlinear Stability](https://mlsanity.com/paper/2103.12263/) (2021)

## Built on by (most cited first, 20 of 20)

- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Robust Classification using Contractive Hamiltonian Neural ODEs](https://mlsanity.com/paper/2203.11805/) (2022)
- [Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach](https://mlsanity.com/paper/2112.05310/) (2021)
- [RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks](https://mlsanity.com/paper/2106.08928/) (2021)
- [Subhomogeneous Deep Equilibrium Models](https://mlsanity.com/paper/2403.00720/) (2024)
- [Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks](https://mlsanity.com/paper/2204.00187/) (2022)
- [Neural-Rendezvous: Provably Robust Guidance and Control to Encounter Interstellar Objects](https://mlsanity.com/paper/2208.04883/) (2022)
- [Lipschitz Multiscale Deep Equilibrium Models: A Theoretically Guaranteed and Accelerated Approach](https://mlsanity.com/paper/2602.03297/) (2026)
- [Bifurcation Models: Learning Set-Valued Solution Maps with Weight-Tied Dynamics](https://mlsanity.com/paper/2605.07277/) (2026)
- [Limitations of Learning Tanh Neural Networks with Finite Precision](https://mlsanity.com/paper/2606.11104/) (2026)
- [Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach](https://mlsanity.com/paper/2208.03889/) (2022)
- [Fixed-Point Centrality for Networks](https://mlsanity.com/paper/2209.07070/) (2022)
- [Ortho-ODE: Enhancing Robustness and of Neural ODEs against Adversarial Attacks](https://mlsanity.com/paper/2305.09179/) (2023)
- [IDKM: Memory Efficient Neural Network Quantization via Implicit, Differentiable k-Means](https://mlsanity.com/paper/2312.07759/) (2023)
- [Robust Convolution Neural ODEs via Contractivity-promoting regularization](https://mlsanity.com/paper/2508.11432/) (2025)
- [A Nonlinear Separation Principle via Contraction Theory: Applications to Neural Networks, Control, and Learning](https://mlsanity.com/paper/2604.15238/) (2026)
- [End-to-End Learning of Safe Optimal Feedback Control in High Dimensions with Control Barrier Function Layers](https://mlsanity.com/paper/2607.20674/) (2026)
- [Learning from the Descent Direction: Adaptive Gradient Descent under One-Sided H"older Regularity](https://mlsanity.com/paper/2607.22906/) (2026)
- [Certified Inference and Training for Deep Equilibrium Networks: A Continuation Framework with Polynomial Complexity Guarantees](https://mlsanity.com/paper/2609.16485/) (2026)
- [Deep learning from a control vantage point](https://mlsanity.com/paper/2610.14385/) (2026)

## Abstract (arXiv)

Implicit neural networks, a.k.a., deep equilibrium networks, are a class of implicit-depth learning models where function evaluation is performed by solving a fixed point equation. They generalize classic feedforward models and are equivalent to infinite-depth weight-tied feedforward networks. While implicit models show improved accuracy and significant reduction in memory consumption, they can suffer from ill-posedness and convergence instability. This paper provides a new framework, which we call Non-Euclidean Monotone Operator Network (NEMON), to design well-posed and robust implicit neural networks based upon contraction theory for the non-Euclidean norm ℓ∞. Our framework includes (i) a novel condition for well-posedness based on one-sided Lipschitz constants, (ii) an average iteration for computing fixed-points, and (iii) explicit estimates on input-output Lipschitz constants. Additionally, we design a training problem with the well-posedness condition and the average iteration as constraints and, to achieve robust models, with the input-output Lipschitz constant as a regularizer. Our ℓ∞ well-posedness condition leads to a larger polytopic training search space than existing conditions and our average iteration enjoys accelerated convergence. Finally, we evaluate our framework in image classification through the MNIST and the CIFAR-10 datasets. Our numerical results demonstrate improved accuracy and robustness of the implicit models with smaller input-output Lipschitz bounds. Code is available at https://github.com/davydovalexander/Non-EuclideanMonOpNet.