# Implicitly Defined Layers in Neural Networks

arXiv: https://arxiv.org/abs/2003.01822 · HTML: https://mlsanity.com/paper/2003.01822/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 7 of 7)

- [JFB: Jacobian-Free Backpropagation for Implicit Networks](https://mlsanity.com/paper/2103.12803/) (2021)
- [Alternating Differentiation for Optimization Layers](https://mlsanity.com/paper/2210.01802/) (2022)
- [Almost Surely Stable Deep Dynamics](https://mlsanity.com/paper/2103.14722/) (2021)
- [Implicit Riemannian Concave Potential Maps](https://mlsanity.com/paper/2110.01288/) (2021)
- [MinBackProp -- Backpropagating through Minimal Solvers](https://mlsanity.com/paper/2404.17993/) (2024)
- [Generalized Laplacian Regularized Framelet Graph Neural Networks](https://mlsanity.com/paper/2210.15092/) (2022)
- [End-to-End Learning-based Operation of Integrated Energy Systems for Buildings and Data Centers](https://mlsanity.com/paper/2604.14184/) (2026)

## Abstract (arXiv)

In conventional formulations of multilayer feedforward neural networks, the individual layers are customarily defined by explicit functions. In this paper we demonstrate that defining individual layers in a neural network implicitly provide much richer representations over the standard explicit one, consequently enabling a vastly broader class of end-to-end trainable architectures. We present a general framework of implicitly defined layers, where much of the theoretical analysis of such layers can be addressed through the implicit function theorem. We also show how implicitly defined layers can be seamlessly incorporated into existing machine learning libraries. In particular with respect to current automatic differentiation techniques for use in backpropagation based training. Finally, we demonstrate the versatility and relevance of our proposed approach on a number of diverse example problems with promising results.