mlsanityData to 7 Oct 2026

Unsupervised Reservoir Computing for Solving Ordinary Differential Equations

arXiv 2108.11417 ↗First posted 25 Aug 2021cs.LG cs.NE physics.comp-phcited by 11

Not on the map. Why?

Abstract (arXiv)

There is a wave of interest in using unsupervised neural networks for solving differential equations. The existing methods are based on feed-forward networks, {while} recurrent neural network differential equation solvers have not yet been reported. We introduce an unsupervised reservoir computing (RC), an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of recurrent neural network outputs without using backpropagation. The internal weights of an RC are fixed, while only a linear output layer is trained, yielding efficient training. However, RC performance strongly depends on finding the optimal hyper-parameters, which is a computationally expensive process. We use Bayesian optimization to efficiently discover optimal sets in a high-dimensional hyper-parameter space and numerically show that one set is robust and can be used to solve an ODE for different initial conditions and time ranges. A closed-form formula for the optimal output weights is derived to solve first order linear equations in a backpropagation-free learning process. We extend the RC approach by solving nonlinear system of ODEs using a hybrid optimization method consisting of gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems of equations demonstrates the efficiency of the RC ODE solver.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 19 of its 46 references that are in the map. First posted on arXiv 25 Aug 2021.

Builds on

19 of 46 references.

References · 19

  1. 2018Neural Ordinary Differential Equationscited by 3,733
  2. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  3. 2019Hamiltonian Neural Networkscited by 702
  4. 2017DGM: A deep learning algorithm for solving partial differential equationscited by 674
  5. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  6. 2019Scalable Global Optimization via Local Bayesian Optimizationcited by 359
  7. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  8. 2017Reversible Architectures for Arbitrarily Deep Residual Neural Networkscited by 69
  9. 2020NVIDIA SimNet^{TM}: an AI-accelerated multi-physics simulation frameworkcited by 58
  10. 2018A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equationscited by 55
  11. 2020Hypersolvers: Toward Fast Continuous-Depth Modelscited by 33
  12. 2020Solving Differential Equations Using Neural Network Solution Bundlescited by 20
7 more
  1. 2019Theory of neuromorphic computing by waves: machine learning by rogue waves, dispersive shocks, and solitonscited by 19
  2. 2020Reservoir Computing meets Recurrent Kernels and Structured Transformscited by 15
  3. 2020Unsupervised Neural Networks for Quantum Eigenvalue Problemscited by 15
  4. 2019Efficient Optimization of Echo State Networks for Time Series Datasetscited by 10
  5. 2020Semi-supervised Neural Networks solve an inverse problem for modeling Covid-19 spreadcited by 8
  6. 2019Bayesian Optimization with Unknown Search Spacecited by 5
  7. 2020Unfolding recurrence by Green's functions for optimized reservoir computingcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 4; 2023: 2; 2024: 1; 2025: 1.

11 papers.

  1. 2023Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behaviorcited by 128
  2. 2021One-Shot Transfer Learning of Physics-Informed Neural Networkscited by 49
  3. 2024PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Lawscited by 30
  4. 2022Physics-Informed Neural Networks for Quantum Eigenvalue Problemscited by 18
  5. 2022Transfer Learning with Physics-Informed Neural Networks for Efficient Simulation of Branched Flowscited by 12
  6. 2021Uncertainty Quantification in Neural Differential Equationscited by 3
  7. 2022RcTorch: a PyTorch Reservoir Computing Package with Automated Hyper-Parameter Optimizationcited by 3
  8. 2022DEQGAN: Learning the Loss Function for PINNs with Generative Adversarial Networkscited by 3
  9. 2021A Meta-learning Approach to Reservoir Computing: Time Series Prediction with Limited Datacited by 1
  10. 2023Unsupervised Random Quantum Networks for PDEscited by 1
  11. 2025Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulencecited by 1