# Deep Neural Network Structures Solving Variational Inequalities

arXiv: https://arxiv.org/abs/1808.07526 · HTML: https://mlsanity.com/paper/1808.07526/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2018.

## Built on by (most cited first, 26 of 26)

- [Nonsmooth Implicit Differentiation for Machine Learning and Optimization](https://mlsanity.com/paper/2106.04350/) (2021)
- [Transductive Zero-Shot and Few-Shot CLIP](https://mlsanity.com/paper/2405.18437/) (2024)
- [Generalized Normalizing Flows via Markov Chains](https://mlsanity.com/paper/2111.12506/) (2021)
- [Connections between Numerical Algorithms for PDEs and Neural Networks](https://mlsanity.com/paper/2107.14742/) (2021)
- [Lifted Bregman Training of Neural Networks](https://mlsanity.com/paper/2208.08772/) (2022)
- [Designing Stable Neural Networks using Convex Analysis and ODEs](https://mlsanity.com/paper/2306.17332/) (2023)
- [Towards Practical Few-Shot Query Sets: Transductive Minimum Description Length Inference](https://mlsanity.com/paper/2210.14545/) (2022)
- [Lipschitz-bounded 1D convolutional neural networks using the Cayley transform and the controllability Gramian](https://mlsanity.com/paper/2303.11835/) (2023)
- [CertViT: Certified Robustness of Pre-Trained Vision Transformers](https://mlsanity.com/paper/2302.10287/) (2023)
- [Fixed points of nonnegative neural networks](https://mlsanity.com/paper/2106.16239/) (2021)
- [Proximal Residual Flows for Bayesian Inverse Problems](https://mlsanity.com/paper/2211.17158/) (2022)
- [Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution](https://mlsanity.com/paper/2110.07202/) (2021)
- [Designing Rotationally Invariant Neural Networks from PDEs and Variational Methods](https://mlsanity.com/paper/2108.13993/) (2021)
- [Quantization Robustness of Monotone Operator Equilibrium Networks](https://mlsanity.com/paper/2603.10562/) (2026)
- [Deep Transform and Metric Learning Networks](https://mlsanity.com/paper/2104.10329/) (2021)
- [Towards Understanding Graph Neural Networks: An Algorithm Unrolling Perspective](https://mlsanity.com/paper/2206.04471/) (2022)
- [An Energy-Based Mechanism for Compositional Behavior](https://mlsanity.com/paper/2512.04745/) (2025)
- [Reframing Neural Networks: Deep Structure in Overcomplete Representations](https://mlsanity.com/paper/2103.05804/) (2021)
- [Stability Bounds for the Unfolded Forward-Backward Algorithm](https://mlsanity.com/paper/2412.17888/) (2024)
- [Circuit realization and hardware linearization of monotone operator equilibrium networks](https://mlsanity.com/paper/2509.13793/) (2025)
- [Variational models for signal processing with Graph Neural Networks](https://mlsanity.com/paper/2103.16337/) (2021)
- [CQnet: convex-geometric interpretation and constraining neural-network trajectories](https://mlsanity.com/paper/2302.10895/) (2023)
- [A lifted Bregman strategy for training unfolded proximal neural network Gaussian denoisers](https://mlsanity.com/paper/2408.08742/) (2024)
- [A Single-Loop Bilevel Deep Learning Method for Optimal Control of Obstacle Problems](https://mlsanity.com/paper/2601.04120/) (2026)
- [Predictive Coding with Bayesian Priors via Proximal Gradients](https://mlsanity.com/paper/2606.08374/) (2026)
- [A Bregman Proximal Viewpoint on Neural Operators](https://mlsanity.com/paper/2610.20120/) (2026)

## Abstract (arXiv)

Motivated by structures that appear in deep neural networks, we investigate nonlinear composite models alternating proximity and affine operators defined on different spaces. We first show that a wide range of activation operators used in neural networks are actually proximity operators. We then establish conditions for the averagedness of the proposed composite constructs and investigate their asymptotic properties. It is shown that the limit of the resulting process solves a variational inequality which, in general, does not derive from a minimization problem.