# Learning continuous models for continuous physics

arXiv: https://arxiv.org/abs/2202.08494 · HTML: https://mlsanity.com/paper/2202.08494/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 73 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 30 of 39)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021)
- [Neural Operator: Learning Maps Between Function Spaces](https://mlsanity.com/paper/2108.08481/) (2021)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Deep Neural Networks Motivated by Partial Differential Equations](https://mlsanity.com/paper/1804.04272/) (2018)
- [Dissecting Neural ODEs](https://mlsanity.com/paper/2002.08071/) (2020)
- [Neural Jump Stochastic Differential Equations](https://mlsanity.com/paper/1905.10403/) (2019)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Linearly-Recurrent Autoencoder Networks for Learning Dynamics](https://mlsanity.com/paper/1712.01378/) (2017)
- [Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data](https://mlsanity.com/paper/2002.06716/) (2020)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [Traditional and Heavy-Tailed Self Regularization in Neural Network Models](https://mlsanity.com/paper/1901.08276/) (2019)
- [Objective Mismatch in Model-based Reinforcement Learning](https://mlsanity.com/paper/2002.04523/) (2020)
- [Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems](https://mlsanity.com/paper/1801.01236/) (2018)
- [Lipschitz Recurrent Neural Networks](https://mlsanity.com/paper/2006.12070/) (2020)
- [PySINDy: A comprehensive Python package for robust sparse system identification](https://mlsanity.com/paper/2111.08481/) (2021)
- [Physics-informed Autoencoders for Lyapunov-stable Fluid Flow Prediction](https://mlsanity.com/paper/1905.10866/) (2019)
- [Learning Compositional Koopman Operators for Model-Based Control](https://mlsanity.com/paper/1910.08264/) (2019)
- [Combating the Compounding-Error Problem with a Multi-step Model](https://mlsanity.com/paper/1905.13320/) (2019)
- [Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks](https://mlsanity.com/paper/1901.08278/) (2019)
- [Noisy Recurrent Neural Networks](https://mlsanity.com/paper/2102.04877/) (2021)
- [Long Expressive Memory for Sequence Modeling](https://mlsanity.com/paper/2110.04744/) (2021)
- [Continuous-in-Depth Neural Networks](https://mlsanity.com/paper/2008.02389/) (2020)
- [Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs](https://mlsanity.com/paper/2006.16210/) (2020)
- [Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers](https://mlsanity.com/paper/2108.00781/) (2021)
- [Bridging Physics-based and Data-driven modeling for Learning Dynamical Systems](https://mlsanity.com/paper/2011.10616/) (2020)
- [Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures](https://mlsanity.com/paper/1910.09045/) (2019)
- [ResNet After All? Neural ODEs and Their Numerical Solution](https://mlsanity.com/paper/2007.15386/) (2020)
- [Post-mortem on a deep learning contest: a Simpson's paradox and the complementary roles of scale metrics versus shape metrics](https://mlsanity.com/paper/2106.00734/) (2021)

## Built on by (most cited first, 28 of 28)

- [Learning Physical Models that Can Respect Conservation Laws](https://mlsanity.com/paper/2302.11002/) (2023)
- [Towards Stability of Autoregressive Neural Operators](https://mlsanity.com/paper/2306.10619/) (2023)
- [Learning differentiable solvers for systems with hard constraints](https://mlsanity.com/paper/2207.08675/) (2022)
- [Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning](https://mlsanity.com/paper/2402.15734/) (2024)
- [Zero-shot forecasting of chaotic systems](https://mlsanity.com/paper/2409.15771/) (2024)
- [OceanNet: A principled neural operator-based digital twin for regional oceans](https://mlsanity.com/paper/2310.00813/) (2023)
- [On Numerical Integration in Neural Ordinary Differential Equations](https://mlsanity.com/paper/2206.07335/) (2022)
- [SuperBench: A Super-Resolution Benchmark Dataset for Scientific Machine Learning](https://mlsanity.com/paper/2306.14070/) (2023)
- [Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning](https://mlsanity.com/paper/2505.11349/) (2025)
- [Elucidating the Design Choice of Probability Paths in Flow Matching for Forecasting](https://mlsanity.com/paper/2410.03229/) (2024)
- [Neural equilibria for long-term prediction of nonlinear conservation laws](https://mlsanity.com/paper/2501.06933/) (2025)
- [Reduced-order modeling for parameterized PDEs via implicit neural representations](https://mlsanity.com/paper/2311.16410/) (2023)
- [Stability-Aware Training of Machine Learning Force Fields with Differentiable Boltzmann Estimators](https://mlsanity.com/paper/2402.13984/) (2024)
- [Learning dynamical systems with hit-and-run random feature maps](https://mlsanity.com/paper/2501.06661/) (2025)
- [Continuous Spatiotemporal Transformers](https://mlsanity.com/paper/2301.13338/) (2023)
- [Stability-Informed Initialization of Neural Ordinary Differential Equations](https://mlsanity.com/paper/2311.15890/) (2023)
- [SeqLink: A Robust Neural-ODE Architecture for Modelling Partially Observed Time Series](https://mlsanity.com/paper/2212.03560/) (2022)
- [Some of the variables, some of the parameters, some of the times, with some physics known: Identification with partial information](https://mlsanity.com/paper/2304.14214/) (2023)
- [Residuals Are Not Enough: Limits of Physics-Informed Pre-Training for Scientific Foundation Models](https://mlsanity.com/paper/2503.19081/) (2025)
- [ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow](https://mlsanity.com/paper/2607.27924/) (2026)
- [Uncertainty-Aware Diagnostics for Physics-Informed Machine Learning](https://mlsanity.com/paper/2510.26121/) (2025)
- [Bayesian identification of nonseparable Hamiltonians with multiplicative noise using deep learning and reduced-order modeling](https://mlsanity.com/paper/2401.12476/) (2024)
- [Neural DAEs: Constrained neural networks](https://mlsanity.com/paper/2211.14302/) (2022)
- [Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization](https://mlsanity.com/paper/2605.29153/) (2026)
- [Noisy Learning for Neural ODEs Acts as a Robustness Locus Widening](https://mlsanity.com/paper/2206.08237/) (2022)
- [Interpretability and Generalization Bounds for Learning Spatial Physics](https://mlsanity.com/paper/2506.15199/) (2025)
- [Reconstructing Multi-Scale Physical Fields from Extremely Sparse Measurements with an Autoencoder-Diffusion Cascade](https://mlsanity.com/paper/2512.01572/) (2025)
- [Eigenanalysis framework for autoregressive neural emulators of multi-scale chaotic dynamics](https://mlsanity.com/paper/2608.16084/) (2026)

## Abstract (arXiv)

Dynamical systems that evolve continuously over time are ubiquitous throughout science and engineering. Machine learning (ML) provides data-driven approaches to model and predict the dynamics of such systems. A core issue with this approach is that ML models are typically trained on discrete data, using ML methodologies that are not aware of underlying continuity properties. This results in models that often do not capture any underlying continuous dynamics -- either of the system of interest, or indeed of any related system. To address this challenge, we develop a convergence test based on numerical analysis theory. Our test verifies whether a model has learned a function that accurately approximates an underlying continuous dynamics. Models that fail this test fail to capture relevant dynamics, rendering them of limited utility for many scientific prediction tasks; while models that pass this test enable both better interpolation and better extrapolation in multiple ways. Our results illustrate how principled numerical analysis methods can be coupled with existing ML training/testing methodologies to validate models for science and engineering applications.