# FMint-SDE: A Multimodal Foundation Model for Accelerating Numerical Simulation of SDEs via Error Correction

arXiv: https://arxiv.org/abs/2510.27173 · HTML: https://mlsanity.com/paper/2510.27173/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 24 of 24)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [Neural SDEs as Infinite-Dimensional GANs](https://mlsanity.com/paper/2102.03657/) (2021)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [In-Context Operator Learning with Data Prompts for Differential Equation Problems](https://mlsanity.com/paper/2304.07993/) (2023)
- [Efficient and Accurate Gradients for Neural SDEs](https://mlsanity.com/paper/2105.13493/) (2021)
- [Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs](https://mlsanity.com/paper/2403.12553/) (2024)
- [Towards a Foundation Model for Partial Differential Equations: Multi-Operator Learning and Extrapolation](https://mlsanity.com/paper/2404.12355/) (2024)
- [Stable Neural Stochastic Differential Equations in Analyzing Irregular Time Series Data](https://mlsanity.com/paper/2402.14989/) (2024)
- [PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics](https://mlsanity.com/paper/2409.09811/) (2024)
- [Computing Committor Functions for the Study of Rare Events Using Deep Learning](https://mlsanity.com/paper/1906.06285/) (2019)
- [UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptation](https://mlsanity.com/paper/2403.07187/) (2024)
- [Black-box Variational Inference for Stochastic Differential Equations](https://mlsanity.com/paper/1802.03335/) (2018)
- [FMint: Bridging Human Designed and Data Pretrained Models for Differential Equation Foundation Model](https://mlsanity.com/paper/2404.14688/) (2024)
- [On Fast Simulation of Dynamical System with Neural Vector Enhanced Numerical Solver](https://mlsanity.com/paper/2208.03680/) (2022)
- [Optimal control for sampling the transition path process and estimating rates](https://mlsanity.com/paper/2305.17112/) (2023)
- [CSRec: Rethinking Sequential Recommendation from A Causal Perspective](https://mlsanity.com/paper/2409.05872/) (2024)
- [The Seven-League Scheme: Deep learning for large time step Monte Carlo simulations of stochastic differential equations](https://mlsanity.com/paper/2009.03202/) (2020)

## Abstract (arXiv)

Fast and accurate simulation of dynamical systems is a fundamental challenge across scientific and engineering domains. Traditional numerical integrators often face a trade-off between accuracy and computational efficiency, while existing neural network-based approaches typically require training a separate model for each case. To overcome these limitations, we introduce a novel multi-modal foundation model for large-scale simulations of differential equations: FMint-SDE (Foundation Model based on Initialization for stochastic differential equations). Based on a decoder-only transformer with in-context learning, FMint-SDE leverages numerical and textual modalities to learn a universal error-correction scheme. It is trained using prompted sequences of coarse solutions generated by conventional solvers, enabling broad generalization across diverse systems. We evaluate our models on a suite of challenging SDE benchmarks spanning applications in molecular dynamics, mechanical systems, finance, and biology. Experimental results show that our approach achieves a superior accuracy-efficiency tradeoff compared to classical solvers, underscoring the potential of FMint-SDE as a general-purpose simulation tool for dynamical systems.