mlsanityData to 7 Oct 2026

A Generalization Bound for a Family of Implicit Networks

arXiv 2410.07427 ↗First posted 9 Oct 2024cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Implicit networks are a class of neural networks whose outputs are defined by the fixed point of a parameterized operator. They have enjoyed success in many applications including natural language processing, image processing, and numerous other applications. While they have found abundant empirical success, theoretical work on its generalization is still under-explored. In this work, we consider a large family of implicit networks defined parameterized contractive fixed point operators. We show a generalization bound for this class based on a covering number argument for the Rademacher complexity of these architectures.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 27 of its 41 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

27 of 41 references.

References · 27

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  3. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  4. 2019Deep Equilibrium Modelscited by 603
  5. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 588
  6. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  7. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  8. 2017Learned Primal-dual Reconstructioncited by 251
  9. 2016A vector-contraction inequality for Rademacher complexitiescited by 141
  10. 2021Deep Equilibrium Architectures for Inverse Problems in Imagingcited by 126
  11. 2019Implicit Deep Learningcited by 124
  12. 2020Monotone operator equilibrium networkscited by 100
15 more
  1. 2021JFB: Jacobian-Free Backpropagation for Implicit Networkscited by 84
  2. 2019On Generalization Bounds of a Family of Recurrent Neural Networkscited by 42
  3. 2018Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterizationcited by 35
  4. 2022Path Independent Equilibrium Models Can Better Exploit Test-Time Computationcited by 33
  5. 2021Learning the optimal Tikhonov regularizer for inverse problemscited by 26
  6. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21
  7. 2021Feasibility-based Fixed Point Networkscited by 17
  8. 2021A global convergence theory for deep ReLU implicit networks via over-parameterizationcited by 17
  9. 2020Compressive Sensing and Neural Networks from a Statistical Learning Perspectivecited by 15
  10. 2022Explainable AI via Learning to Optimizecited by 14
  11. 2023Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Modelscited by 14
  12. 2023Differentiating Through Integer Linear Programs with Quadratic Regularization and Davis-Yin Splittingcited by 11
  13. 2024On Logical Extrapolation for Mazes with Recurrent and Implicit Networkscited by 7
  14. 2023On the Generalization and Approximation Capacities of Neural Controlled Differential Equationscited by 3
  15. 2022On the optimization and generalization of overparameterized implicit neural networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025End-to-End Training of High-Dimensional Optimal Control with Implicit Hamiltonians via Jacobian-Free Backpropagationcited by 3
  2. 2025Expressive Power of Implicit Models: Rich Equilibria and Test-Time Scalingcited by 1
  3. 2026On the Convergence of Jacobian-Free Backpropagation for Optimal Control Problems with Implicit Hamiltonianscited by 1
  4. 2026Upper Generalization Bounds for Neural Oscillatorscited by 0