mlsanityData to 14 Oct 2026

Predicting Ordinary Differential Equations with Transformers

arXiv 2307.12617 ↗First posted 24 Jul 2023cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

We develop a transformer-based sequence-to-sequence model that recovers scalar ordinary differential equations (ODEs) in symbolic form from irregularly sampled and noisy observations of a single solution trajectory. We demonstrate in extensive empirical evaluations that our model performs better or on par with existing methods in terms of accurate recovery across various settings. Moreover, our method is efficiently scalable: after one-time pretraining on a large set of ODEs, we can infer the governing law of a new observed solution in a few forward passes of the model.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 37 of its 75 references that are in the map. First posted on arXiv 24 Jul 2023.

Builds on

37 of 75 references.

References · 37

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  6. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  7. 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 309
  8. 2019Deep Learning for Symbolic Mathematicscited by 270
  9. 2021Contemporary Symbolic Regression Methods and their Relative Performancecited by 215
  10. 2018PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Networkcited by 210
  11. 2022End-to-end symbolic regression with transformerscited by 203
  12. 2021Neural Symbolic Regression that Scalescited by 192
25 more
  1. 2018Learning Equations for Extrapolation and Controlcited by 156
  2. 2020AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularitycited by 144
  3. 2021SymbolicGPT: A Generative Transformer Model for Symbolic Regressioncited by 105
  4. 2022Interpretable Scientific Discovery with Symbolic Regression: A Reviewcited by 105
  5. 2021Symbolic Regression via Neural-Guided Genetic Programming Population Seedingcited by 101
  6. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 88
  7. 2020Multiscale Simulations of Complex Systems by Learning their Effective Dynamicscited by 86
  8. 2019Bayesian Symbolic Regressioncited by 78
  9. 2022SymFormer: End-to-end symbolic regression using transformer-based architecturecited by 59
  10. 2022Discovering Governing Equations from Partial Measurements with Deep Delay Autoencoderscited by 58
  11. 2022Deep Symbolic Regression for Recurrent Sequencescited by 54
  12. 2020Learning continuous-time PDEs from sparse data with graph neural networkscited by 53
  13. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 42
  14. 2020Probabilistic Grammars for Equation Discoverycited by 36
  15. 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 36
  16. 2018Learning concise representations for regression by evolving networks of treescited by 33
  17. 2022Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systemscited by 33
  18. 2019Epsilon-Lexicase Selection for Regressioncited by 31
  19. 2021DySMHO: Data-Driven Discovery of Governing Equations for Dynamical Systems via Moving Horizon Optimizationcited by 19
  20. 2019Data-driven discovery of free-form governing differential equationscited by 16
  21. 2022GSR: A Generalized Symbolic Regression Approachcited by 11
  22. 2022Recurrent Neural Networks for Dynamical Systems: Applications to Ordinary Differential Equations, Collective Motion, and Hydrological Modelingcited by 10
  23. 2019Equation Discovery for Nonlinear System Identificationcited by 7
  24. 2021Inferring the Structure of Ordinary Differential Equationscited by 7
  25. 2020RODE-Net: Learning Ordinary Differential Equations with Randomness from Datacited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 9; 2026: 8.

26 papers.

Sort:

  1. 2023ODEFormer: Symbolic Regression of Dynamical Systems with Transformerscited by 62
  2. 2023SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-trainingcited by 47
  3. 2024Recent Advances on Machine Learning for Computational Fluid Dynamics: A Surveycited by 28
  4. 2024Marrying Causal Representation Learning with Dynamical Systems for Sciencecited by 23
  5. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 21
  6. 2023PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformerscited by 16
  7. 2024Zero-shot Imputation with Foundation Inference Models for Dynamical Systemscited by 13
  8. 2025Panda: A pretrained forecast model for chaotic dynamicscited by 13
  9. 2025Identifiability Challenges in Sparse Linear Ordinary Differential Equationscited by 7
  10. 2025Grammar-based Ordinary Differential Equation Discoverycited by 6
  11. 2025Adaptive parameters identification for nonlinear dynamics using deep permutation invariant networkscited by 5
  12. 2025Discovering Physics Laws of Dynamical Systems via Invariant Function Learningcited by 5
14 more
  1. 2025Structured Kolmogorov-Arnold Neural ODEs for Interpretable Learning and Symbolic Discovery of Nonlinear Dynamicscited by 4
  2. 2024Decoding complexity: how machine learning is redefining scientific discoverycited by 3
  3. 2024Active Symbolic Discovery of Ordinary Differential Equations via Phase Portrait Sketchingcited by 3
  4. 2026Foundation Inference Models for Ordinary Differential Equationscited by 3
  5. 2025Data-driven Discovery of Digital Twins in Biomedical Researchcited by 2
  6. 2025Predicting symbolic ODEs from multiple trajectoriescited by 2
  7. 2026Neuro-Symbolic ODE Discovery with Latent Grammar Flowcited by 2
  8. 2025Protein Folding with Neural Ordinary Differential Equationscited by 0
  9. 2026OrthoReg: Orthogonal Regularization for Hybrid Symbolic-Neural Dynamical Systemscited by 0
  10. 2026Dualformer: Efficient Feature Extractor for Complex-valued Blind Communication Signal Analysiscited by 0
  11. 2026Symbolic Neural ODEs: Learning interpretable models from time-series datacited by 0
  12. 2026Data-Driven Learning of Unknown Nonlinear Differential Equations Using Functional Analysiscited by 0
  13. 2026FluxDisco: Symbolic Regression for Stoichiometric Dynamical Systems via Monte Carlo Graph Searchcited by 0
  14. 2026Identifying ODEs from Unstructured Data with Causal Representation Learningcited by 0