# Neural oscillators for generalization of physics-informed machine learning

arXiv: https://arxiv.org/abs/2308.08989 · HTML: https://mlsanity.com/paper/2308.08989/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 48 references that are in the map.
- First posted on arXiv 17 Aug 2023.

## Builds on (most cited first, 30 of 33)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism](https://mlsanity.com/paper/2009.04544/) (2020)
- [Respecting causality is all you need for training physics-informed neural networks](https://mlsanity.com/paper/2203.07404/) (2022)
- [Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN](https://mlsanity.com/paper/1803.04831/) (2018)
- [Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling](https://mlsanity.com/paper/1710.11431/) (2017)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs](https://mlsanity.com/paper/2006.16144/) (2020)
- [AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks](https://mlsanity.com/paper/1902.09689/) (2019)
- [Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating a class of inverse problems for PDEs](https://mlsanity.com/paper/2007.01138/) (2020)
- [Full-Capacity Unitary Recurrent Neural Networks](https://mlsanity.com/paper/1611.00035/) (2016)
- [Coupled Oscillatory Recurrent Neural Network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies](https://mlsanity.com/paper/2010.00951/) (2020)
- [Learning in High Dimension Always Amounts to Extrapolation](https://mlsanity.com/paper/2110.09485/) (2021)
- [UnICORNN: A recurrent model for learning very long time dependencies](https://mlsanity.com/paper/2103.05487/) (2021)
- [Machine learning structure preserving brackets for forecasting irreversible processes](https://mlsanity.com/paper/2106.12619/) (2021)
- [LNO: Laplace Neural Operator for Solving Differential Equations](https://mlsanity.com/paper/2303.10528/) (2023)
- [DPM: A Novel Training Method for Physics-Informed Neural Networks in Extrapolation](https://mlsanity.com/paper/2012.02681/) (2020)
- [Rethinking Data Augmentation for Single-source Domain Generalization in Medical Image Segmentation](https://mlsanity.com/paper/2211.14805/) (2022)
- [Long Expressive Memory for Sequence Modeling](https://mlsanity.com/paper/2110.04744/) (2021)
- [Recurrent Orthogonal Networks and Long-Memory Tasks](https://mlsanity.com/paper/1602.06662/) (2016)
- [On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs](https://mlsanity.com/paper/2004.01806/) (2020)
- [Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics](https://mlsanity.com/paper/1905.12080/) (2019)
- [Physics-informed neural networks for solving forward and inverse problems in complex beam systems](https://mlsanity.com/paper/2303.01055/) (2023)
- [Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning](https://mlsanity.com/paper/2212.05301/) (2022)
- [NeurOLight: A Physics-Agnostic Neural Operator Enabling Parametric Photonic Device Simulation](https://mlsanity.com/paper/2209.10098/) (2022)

## Built on by (most cited first, 9 of 9)

- [Transfer learning for improved generalizability in causal physics-informed neural networks for beam simulations](https://mlsanity.com/paper/2311.00578/) (2023)
- [Magnetic Hysteresis Modeling with Neural Operators](https://mlsanity.com/paper/2407.03261/) (2024)
- [Neural oscillators for magnetic hysteresis modeling](https://mlsanity.com/paper/2308.12002/) (2023)
- [Upper Approximation Bounds for Neural Oscillators](https://mlsanity.com/paper/2512.01015/) (2025)
- [DiffGrad for Physics-Informed Neural Networks](https://mlsanity.com/paper/2409.03239/) (2024)
- [REAct: Rational Exponential Activation for Better Learning and Generalization in PINNs](https://mlsanity.com/paper/2503.02267/) (2025)
- [Process-Informed Forecasting of Complex Thermal Dynamics in Pharmaceutical Manufacturing](https://mlsanity.com/paper/2509.20349/) (2025)
- [Physics-Informed Spiking Neural Networks via Conservative Flux Quantization](https://mlsanity.com/paper/2511.21784/) (2025)
- [Upper Generalization Bounds for Neural Oscillators](https://mlsanity.com/paper/2603.09742/) (2026)

## Abstract (arXiv)

A primary challenge of physics-informed machine learning (PIML) is its generalization beyond the training domain, especially when dealing with complex physical problems represented by partial differential equations (PDEs). This paper aims to enhance the generalization capabilities of PIML, facilitating practical, real-world applications where accurate predictions in unexplored regions are crucial. We leverage the inherent causality and temporal sequential characteristics of PDE solutions to fuse PIML models with recurrent neural architectures based on systems of ordinary differential equations, referred to as neural oscillators. Through effectively capturing long-time dependencies and mitigating the exploding and vanishing gradient problem, neural oscillators foster improved generalization in PIML tasks. Extensive experimentation involving time-dependent nonlinear PDEs and biharmonic beam equations demonstrates the efficacy of the proposed approach. Incorporating neural oscillators outperforms existing state-of-the-art methods on benchmark problems across various metrics. Consequently, the proposed method improves the generalization capabilities of PIML, providing accurate solutions for extrapolation and prediction beyond the training data.