mlsanityData to 7 Oct 2026

A Unified Representation of Neural Networks Architectures

arXiv 2512.17593 ↗First posted 19 Dec 2025cs.LG math.OCcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper we consider the limiting case of neural networks (NNs) architectures when the number of neurons in each hidden layer and the number of hidden layers tend to infinity thus forming a continuum, and we derive approximation errors as a function of the number of neurons and/or hidden layers. Firstly, we consider the case of neural networks with a single hidden layer and we derive an integral infinite width neural representation that generalizes existing continuous neural networks (CNNs) representations. Then we extend this to deep residual CNNs that have a finite number of integral hidden layers and residual connections. Secondly, we revisit the relation between neural ODEs and deep residual NNs and we formalize approximation errors via discretization techniques. Then, we merge these two approaches into a unified homogeneous representation of NNs as a Distributed Parameter neural Network (DiPaNet) and we show that most of the existing finite and infinite-dimensional NNs architectures are related via homogenization/discretization with the DiPaNet representation. Our approach is purely deterministic and applies to general, uniformly continuous matrix weight functions. Relations with neural fields and other neural integro-differential equations are discussed along with further possible generalizations and applications of the DiPaNet framework.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 32 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

16 of 32 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Neural Ordinary Differential Equationscited by 3,733
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2014Deep Learning in Neural Networks: An Overviewcited by 765
  5. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  6. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  7. 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
  8. 2018Small ReLU networks are powerful memorizers: a tight analysis of memorization capacitycited by 76
  9. 2020A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networkscited by 68
  10. 2020Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllerscited by 42
  11. 2017Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systemscited by 39
  12. 2022Neural Integro-Differential Equationscited by 17
4 more
  1. 2020l2 Induced Norm Analysis of Discrete-Time LTI Systems for Nonnegative Input Signals and Its Application to Stability Analysis of Recurrent Neural Networkscited by 5
  2. 2021Neural Field Models: A mathematical overview and unifying frameworkcited by 2
  3. 2023Adaptive observer and control of spatiotemporal delayed neural fieldscited by 1
  4. 2025Towards an Optimal Control Perspective of ResNet Trainingcited by 1

Built on by

Not cited yet.