mlsanityData to 7 Oct 2026

Infinitely deep neural networks as diffusion processes

arXiv 1905.11065 ↗First posted 27 May 2019stat.ML cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

When the parameters are independently and identically distributed (initialized) neural networks exhibit undesirable properties that emerge as the number of layers increases, e.g. a vanishing dependency on the input and a concentration on restrictive families of functions including constant functions. We consider parameter distributions that shrink as the number of layers increases in order to recover well-behaved stochastic processes in the limit of infinite depth. This leads to set forth a link between infinitely deep residual networks and solutions to stochastic differential equations, i.e. diffusion processes. We show that these limiting processes do not suffer from the aforementioned issues and investigate their properties.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 6; 2023: 3; 2024: 2; 2025: 3; 2026: 2.

21 papers.

Sort:

  1. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
  2. 2022On the infinite-depth limit of finite-width neural networkscited by 22
  3. 2023Width and Depth Limits Commute in Residual Networkscited by 21
  4. 2025SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equationscited by 19
  5. 2022Scaling ResNets in the Large-depth Regimecited by 18
  6. 2021Scaling Properties of Deep Residual Networkscited by 16
  7. 2024Clustering in pure-attention hardmax transformers and its role in sentiment analysiscited by 15
  8. 2023Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Modelscited by 14
  9. 2022Fractional SDE-Net: Generation of Time Series Data with Long-term Memorycited by 9
  10. 2021Continuous-Depth Neural Models for Dynamic Graph Predictioncited by 7
  11. 2024Rademacher Complexity of Neural ODEs via Chen-Fliess Seriescited by 5
  12. 2026Clustering in Deep Stochastic Transformerscited by 5
9 more
  1. 2022Asymptotic Analysis of Deep Residual Networkscited by 3
  2. 2021Learning Stochastic Optimal Policies via Gradient Descentcited by 2
  3. 2022Nonlinear controllability and function representation by neural stochastic differential equationscited by 2
  4. 2023Commutative Width and Depth Scaling in Deep Neural Networkscited by 1
  5. 2021Neural Options Pricingcited by 0
  6. 2022Neural Langevin Dynamics: towards interpretable Neural Stochastic Differential Equationscited by 0
  7. 2025Neural Mean-Field Games: Extending Mean-Field Game Theory with Neural Stochastic Differential Equationscited by 0
  8. 2025Feature Learning Dynamics in Infinite-Depth Neural Networkscited by 0
  9. 2026Collective Kernel EFT for Pre-activation ResNetscited by 0