mlsanityData to 7 Oct 2026

Upper Generalization Bounds for Neural Oscillators

arXiv 2603.09742 ↗First posted 10 Mar 2026cs.LG math.DS stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural oscillators that originate from second-order ordinary differential equations (ODEs) have shown competitive performance in learning mappings between dynamic loads and responses of complex nonlinear structural systems. Despite this empirical success, theoretically quantifying the generalization capacities of their neural network architectures remains undeveloped. In this study, the neural oscillator consisting of a second-order ODE followed by a multilayer perceptron (MLP) is considered. Its upper probably approximately correct (PAC) generalization bound for approximating causal and uniformly continuous operators between continuous temporal function spaces and that for approximating the uniformly asymptotically incrementally stable second-order dynamical systems are derived by leveraging the Rademacher complexity framework. These bounds are further extended to the squared Wasserstein-1 distances between the probability measures of quantities of interest calculated from target causal operators and the corresponding learned neural oscillators. The theoretical results show that the estimation errors grow polynomially with respect to both MLP sizes and the time length, thereby avoiding the curse of parametric complexity. Furthermore, the derived error bounds demonstrate that constraining the Lipschitz constants of the MLPs via loss function regularization can improve the generalization ability of the neural oscillator. Numerical studies considering a Bouc-Wen nonlinear system under stochastic seismic excitation validates the theoretically predicted power laws of the estimation errors with respect to the sample size and time length, and confirms the effectiveness of constraining MLPs' matrix and vector norms in enhancing the performance of the neural oscillator under limited training data.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 59 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

41 of 59 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  8. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  9. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  10. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  11. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  12. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
29 more
  1. 2016A vector-contraction inequality for Rademacher complexitiescited by 141
  2. 2019Robust Learning with Jacobian Regularizationcited by 127
  3. 2017Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activationscited by 104
  4. 2018Echo state networks are universalcited by 86
  5. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  6. 2020Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependenciescited by 81
  7. 2019Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentationcited by 71
  8. 2021UnICORNN: A recurrent model for learning very long time dependenciescited by 56
  9. 2020Approximation Bounds for Random Neural Networks and Reservoir Systemscited by 52
  10. 2019On Generalization Bounds of a Family of Recurrent Neural Networkscited by 42
  11. 2017Universal discrete-time reservoir computers with stochastic inputs and linear readouts using non-homogeneous state-affine systemscited by 39
  12. 2023Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvaluescited by 35
  13. 2024Oscillatory State-Space Modelscited by 35
  14. 2021Framing RNN as a kernel method: A neural ODE approachcited by 23
  15. 2019Risk bounds for reservoir computingcited by 22
  16. 2023Generalization bounds for neural ordinary differential equations and deep residual networkscited by 21
  17. 2023Neural Oscillators are Universalcited by 16
  18. 2020Learning Recurrent Neural Net Models of Nonlinear Systemscited by 11
  19. 2024From Generalization Analysis to Optimization Designs for State Space Modelscited by 11
  20. 2023Neural oscillators for generalization of physics-informed machine learningcited by 9
  21. 2024Operator Learning of Lipschitz Operators: An Information-Theoretic Perspectivecited by 9
  22. 2024The Sample Complexity of Learning Lipschitz Operators with respect to Gaussian Measurescited by 8
  23. 2024Input-to-State Stable Coupled Oscillator Networks for Closed-form Model-based Control in Latent Spacecited by 7
  24. 2024Rademacher Complexity of Neural ODEs via Chen-Fliess Seriescited by 5
  25. 2025Generalization Error Analysis for Selective State-Space Models Through the Lens of Attentioncited by 5
  26. 2024A Generalization Bound for a Family of Implicit Networkscited by 4
  27. 2024Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraintscited by 2
  28. 2024Universality of reservoir systems with recurrent neural networkscited by 1
  29. 2025Upper Approximation Bounds for Neural Oscillatorscited by 1

Built on by

Not cited yet.