mlsanityData to 14 Oct 2026

Machine Learning from a Continuous Viewpoint

arXiv 1912.12777 ↗First posted 30 Dec 2019math.NA cs.NA math.OCcited by 50

Not on the map. Why?

Abstract (arXiv)

We present a continuous formulation of machine learning, as a problem in the calculus of variations and differential-integral equations, in the spirit of classical numerical analysis. We demonstrate that conventional machine learning models and algorithms, such as the random feature model, the two-layer neural network model and the residual neural network model, can all be recovered (in a scaled form) as particular discretizations of different continuous formulations. We also present examples of new models, such as the flow-based random feature model, and new algorithms, such as the smoothed particle method and spectral method, that arise naturally from this continuous formulation. We discuss how the issues of generalization error and implicit regularization can be studied under this framework.

Key facts. Cited by 50 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 17; 2022: 10; 2023: 6; 2024: 6; 2025: 5; 2026: 6.

50 papers.

Sort:

By area:

  1. 2022Overview frequency principle/spectral bias in deep learningcited by 69
  2. 2021Noisy Recurrent Neural Networkscited by 42
  3. 2021SPINN: Sparse, Physics-based, and partially Interpretable Neural Networks for PDEscited by 32
  4. 2021Overparameterization of deep ResNet: zero loss and mean-field analysiscited by 24
  5. 2024Operator Learning Using Random Features: A Tool for Scientific Computingcited by 21
  6. 2023PottsMGNet: A Mathematical Explanation of Encoder-Decoder Based Neural Networkscited by 17
  7. 2021Analysis of feature learning in weight-tied autoencoders via the mean field lenscited by 14
  8. 2021Exact Gap between Generalization Error and Uniform Convergence in Random Feature Modelscited by 13
  9. 2022On Feature Learning in Neural Networks with Global Convergence Guaranteescited by 13
  10. 2023Model-free tracking control of complex dynamical trajectories with machine learningcited by 13
  11. 2021On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regimecited by 12
  12. 2021Subspace Decomposition based DNN algorithm for elliptic type multi-scale PDEscited by 10
38 more
  1. 2022Defects of Convolutional Decoder Networks in Frequency Representationcited by 10
  2. 2022Provably convergent quasistatic dynamics for mean-field two-player zero-sum gamescited by 9
  3. 2021Generalization Error of GAN from the Discriminator's Perspectivecited by 8
  4. 2022On the symmetries in the dynamics of wide two-layer neural networkscited by 8
  5. 2022A Mathematical Framework for Learning Probability Distributionscited by 7
  6. 2023Stochastic Modified Flows, Mean-Field Limits and Dynamics of Stochastic Gradient Descentcited by 7
  7. 2022A blob method for inhomogeneous diffusion with applications to multi-agent control and samplingcited by 6
  8. 2023A convergence result of a continuous model of deep learning via a Lojasiewicz--Simon inequalitycited by 5
  9. 2024Learning Domain-Independent Green's Function For Elliptic Partial Differential Equationscited by 5
  10. 2021Frequency Principle in Deep Learning Beyond Gradient-descent-based Trainingcited by 4
  11. 2021An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Networkcited by 4
  12. 2022Optimal bump functions for shallow ReLU networks: Weight decay, depth separation and the curse of dimensionalitycited by 4
  13. 2024A Mathematical Explanation of UNetcited by 4
  14. 2021A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernelcited by 3
  15. 2022Conservative SPDEs as fluctuating mean field limits of stochastic gradient descentcited by 3
  16. 2023Minimum norm interpolation by perceptra: Explicit regularization and implicit biascited by 3
  17. 2023Efficient Learning of Long-Range and Equivariant Quantum Systemscited by 3
  18. 2024Generalization of Scaled Deep ResNets in the Mean-Field Regimecited by 3
  19. 2024A rationale from frequency perspective for grokking in training neural networkcited by 3
  20. 2021Toward Understanding Convolutional Neural Networks from Volterra Convolution Perspectivecited by 2
  21. 2025A Mathematical Explanation of Transformerscited by 2
  22. 2021Data-informed Deep Optimizationcited by 1
  23. 2021A Riemannian Mean Field Formulation for Two-layer Neural Networks with Batch Normalizationcited by 1
  24. 2024Accurate Learning of Equivariant Quantum Systems from a Single Ground Statecited by 1
  25. 2025Scalable Complexity Control Facilitates Reasoning Ability of LLMscited by 1
  26. 2025FW-GAN: Frequency-Driven Handwriting Synthesis with Wave-Modulated MLP Generatorcited by 1
  27. 2026Spectral Gating Networkscited by 1
  28. 2026Structure-Aware Variational Learning of a Class of Generalized Diffusionscited by 1
  29. 2021Deep limits and cut-off phenomena for neural networkscited by 0
  30. 2021Shallow Representation is Deep: Learning Uncertainty-aware and Worst-case Random Feature Dynamicscited by 0
  31. 2021On Procedural Adversarial Noise Attack And Defensecited by 0
  32. 2022Selected aspects of complex, hypercomplex and fuzzy neural networkscited by 0
  33. 2025Accelerating Particle-based Energetic Variational Inferencecited by 0
  34. 2025Is the Frequency Principle always valid?cited by 0
  35. 2026Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Methodcited by 0
  36. 2026Deep-layer limit and stability analysis of the basic forward-backward-splitting induced network (II): learning problemscited by 0
  37. 2026Born Discrete, Made Smooth: Variational Formulation of Shallow Neural Networkscited by 0
  38. 2026SpiS-GAN: Spiral-Modulated Handwriting Synthesis with Star Operationcited by 0