mlsanityData to 7 Oct 2026

Upper Approximation Bounds for Neural Oscillators

arXiv 2512.01015 ↗First posted 30 Nov 2025cs.LG math.DS math.FAcited by 1

Not on the map. Why?

Abstract (arXiv)

Neural oscillators, originating from second-order ordinary differential equations (ODEs), have demonstrated strong performance in stably learning causal mappings between long-term sequences or continuous temporal functions, as well as in accurately approximating physical systems. However, theoretically quantifying the capacities of their neural network architectures remains a significant challenge. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper approximation bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating uniformly asymptotically incrementally stable second-order dynamical systems are derived. The established proof method of the approximation bound for approximating the causal continuous operators can also be directly applied to state-space models consisting of a linear time-continuous complex recurrent neural network followed by an MLP. Theoretical results reveal that the approximation error of the neural oscillator for approximating the second-order dynamical systems scales polynomially with the reciprocals of the widths of two utilized MLPs, thus overcoming the curse of parametric complexity. The convergence rates of two established approximation error bounds are validated through four numerical cases. These results provide a robust theoretical foundation for the effective application of the neural oscillator in science and engineering.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 66 references that are in the map. First posted on arXiv 30 Nov 2025.

Builds on

41 of 66 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  7. 2018Neural Ordinary Differential Equationscited by 3,733
  8. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  9. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  10. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  11. 2016Error bounds for approximations with deep ReLU networkscited by 567
  12. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
29 more
  1. 2017Approximating Continuous Functions by ReLU Nets of Minimal Widthcited by 120
  2. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  3. 2018Echo state networks are universalcited by 86
  4. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  5. 2020Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependenciescited by 81
  6. 2024Theoretical Foundations of Deep Selective State-Space Modelscited by 73
  7. 2020On Second Order Behaviour in Augmented Neural ODEscited by 68
  8. 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
  9. 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
  10. 2020Approximation Bounds for Random Neural Networks and Reservoir Systemscited by 52
  11. 2021Characterization of the Variation Spaces Corresponding to Shallow Neural Networkscited by 48
  12. 2023Operator learning with PCA-Net: upper and lower complexity boundscited by 44
  13. 2021Universal Approximation Theorems for Differentiable Geometric Deep Learningcited by 41
  14. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  15. 2017Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systemscited by 39
  16. 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memorycited by 38
  17. 2021Elementary superexpressive activationscited by 35
  18. 2023Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvaluescited by 35
  19. 2024Oscillatory State-Space Modelscited by 35
  20. 2023Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscited by 33
  21. 2023The Parametric Complexity of Operator Learningcited by 27
  22. 2023Deep Operator Network Approximation Rates for Lipschitz Operatorscited by 24
  23. 2023Neural Oscillators are Universalcited by 16
  24. 2024Mixtures of Neural Operators Reduce Active Complexity in Operator Learningcited by 16
  25. 2020Learning Recurrent Neural Net Models of Nonlinear Systemscited by 11
  26. 2023Neural oscillators for generalization of physics-informed machine learningcited by 9
  27. 2024Operator Learning of Lipschitz Operators: An Information-Theoretic Perspectivecited by 9
  28. 2024Provable Benefits of Complex Parameterizations for Structured State Space Modelscited by 9
  29. 2024Input-to-State Stable Coupled Oscillator Networks for Closed-form Model-based Control in Latent Spacecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Upper Generalization Bounds for Neural Oscillatorscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Stability and Discretization Error of State Space Model Neural Operators
  2. 2026Oscillatory State-Space Models as Inductive Biases for Physics-Informed Neural PDE Solvers
  3. 2025Observability conditions for neural state-space models with eigenvalues and their roots of unity
  4. 2022Neural Ordinary Differential Equations for Nonlinear System Identification
  5. 2025Structured Kolmogorov-Arnold Neural ODEs for Interpretable Learning and Symbolic Discovery of Nonlinear Dynamics
  6. 2026Time Without Timesteps: Simulating Coupled Dynamical Systems via Self-Consistency
  7. 2022PAC-Bayesian-Like Error Bound for a Class of Linear Time-Invariant Stochastic State-Space Models
  8. 2026Autoregression-Free Neural Operators for Time-Dependent PDEs