# Foundation Inference Models for Ordinary Differential Equations

arXiv: https://arxiv.org/abs/2602.08733 · HTML: https://mlsanity.com/paper/2602.08733/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 73 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 30 of 40)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [The frontier of simulation-based inference](https://mlsanity.com/paper/1911.01429/) (2019)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [ForecastPFN: Synthetically-Trained Zero-Shot Forecasting](https://mlsanity.com/paper/2311.01933/) (2023)
- [Amortized Inference for Causal Structure Learning](https://mlsanity.com/paper/2205.12934/) (2022)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- [Learning unknown ODE models with Gaussian processes](https://mlsanity.com/paper/1803.04303/) (2018)
- [Bayesian Neural Ordinary Differential Equations](https://mlsanity.com/paper/2012.07244/) (2020)
- [Neural graphical modelling in continuous-time: consistency guarantees and algorithms](https://mlsanity.com/paper/2105.02522/) (2021)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Predicting Ordinary Differential Equations with Transformers](https://mlsanity.com/paper/2307.12617/) (2023)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [Dual Parameterization of Sparse Variational Gaussian Processes](https://mlsanity.com/paper/2111.03412/) (2021)
- [An Ode to an ODE](https://mlsanity.com/paper/2006.11421/) (2020)
- [Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models](https://mlsanity.com/paper/2410.09385/) (2024)
- [Data-driven discovery of free-form governing differential equations](https://mlsanity.com/paper/1910.05117/) (2019)
- [Variational multiple shooting for Bayesian ODEs with Gaussian processes](https://mlsanity.com/paper/2106.10905/) (2021)
- [A toolkit for data-driven discovery of governing equations in high-noise regimes](https://mlsanity.com/paper/2111.04870/) (2021)
- [Scalable Inference in SDEs by Direct Matching of the Fokker-Planck-Kolmogorov Equation](https://mlsanity.com/paper/2110.15739/) (2021)
- [Neural Structure Learning with Stochastic Differential Equations](https://mlsanity.com/paper/2311.03309/) (2023)
- [Zero-shot Imputation with Foundation Inference Models for Dynamical Systems](https://mlsanity.com/paper/2402.07594/) (2024)
- [Foundation Inference Models for Markov Jump Processes](https://mlsanity.com/paper/2406.06419/) (2024)
- [Variational Gaussian Process Diffusion Processes](https://mlsanity.com/paper/2306.02066/) (2023)
- [In-Context Learning of Temporal Point Processes with Foundation Inference Models](https://mlsanity.com/paper/2509.24762/) (2025)

## Built on by (most cited first, 3 of 3)

- [EVIL: Evolving Interpretable Algorithms for Zero-Shot Inference on Event Sequences and Time Series with LLMs](https://mlsanity.com/paper/2604.15787/) (2026)
- [Prior-Fitted Functional Flow: In-Context Generative Models for Pharmacokinetics](https://mlsanity.com/paper/2604.17670/) (2026)
- [Continuous-Time Machine Learning: A Unified Mathematical Perspective](https://mlsanity.com/paper/2609.16710/) (2026)

## Abstract (arXiv)

Ordinary differential equations (ODEs) are central to scientific modelling, but inferring their vector fields from noisy trajectories remains challenging. Current approaches such as symbolic regression, Gaussian process (GP) regression, and Neural ODEs often require complex training pipelines and substantial machine learning expertise, or they depend strongly on system-specific prior knowledge. We propose FIM-ODE, a pretrained Foundation Inference Model that amortises low-dimensional ODE inference by predicting the vector field directly from noisy trajectory data in a single forward pass. We pretrain FIM-ODE on a prior distribution over ODEs with low-degree polynomial vector fields and represent the target field with neural operators. FIM-ODE achieves strong zero-shot performance, matching and often improving upon ODEFormer, a recent pretrained symbolic baseline, across a range of regimes despite using a simpler pretraining prior distribution. Pretraining also provides a strong initialisation for finetuning, enabling fast and stable adaptation that outperforms modern neural and GP baselines without requiring machine learning expertise.