# Towards Infinitely Long Neural Simulations: Self-Refining Neural Surrogate Models for Dynamical Systems

arXiv: https://arxiv.org/abs/2603.17750 · HTML: https://mlsanity.com/paper/2603.17750/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Neural General Circulation Models for Weather and Climate](https://mlsanity.com/paper/2311.07222/) (2023)
- [PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers](https://mlsanity.com/paper/2308.05732/) (2023)
- [Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers](https://mlsanity.com/paper/2007.00016/) (2020)
- [Conditional Image Generation with Score-Based Diffusion Models](https://mlsanity.com/paper/2111.13606/) (2021)
- [GenCast: Diffusion-based ensemble forecasting for medium-range weather](https://mlsanity.com/paper/2312.15796/) (2023)
- [Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models](https://mlsanity.com/paper/2504.12626/) (2025)
- [Backpropagation Algorithms and Reservoir Computing in Recurrent Neural Networks for the Forecasting of Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/1910.05266/) (2019)
- [DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting](https://mlsanity.com/paper/2306.01984/) (2023)
- [Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulation](https://mlsanity.com/paper/2309.01745/) (2023)
- [Learning to correct spectral methods for simulating turbulent flows](https://mlsanity.com/paper/2207.00556/) (2022)
- [Training neural operators to preserve invariant measures of chaotic attractors](https://mlsanity.com/paper/2306.01187/) (2023)
- [Challenges of learning multi-scale dynamics with AI weather models: Implications for stability and one solution](https://mlsanity.com/paper/2304.07029/) (2023)
- [DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems](https://mlsanity.com/paper/2402.04467/) (2024)
- [Samudra: An AI Global Ocean Emulator for Climate](https://mlsanity.com/paper/2412.03795/) (2024)
- [Thermalizer: Stable autoregressive neural emulation of spatiotemporal chaos](https://mlsanity.com/paper/2503.18731/) (2025)
- [Generative emulation of chaotic dynamics with coherent prior](https://mlsanity.com/paper/2504.14264/) (2025)
- [Defining error accumulation in ML atmospheric simulators](https://mlsanity.com/paper/2405.14714/) (2024)

## Built on by (most cited first, 1 of 1)

- [Staying on the Attractor: Supervising Neural Surrogates of 3D Turbulence Where They Leave It](https://mlsanity.com/paper/2609.32864/) (2026)

## Abstract (arXiv)

Recent advances in autoregressive neural surrogate models have enabled orders-of-magnitude speedups in simulating dynamical systems. However, autoregressive models are generally prone to distribution drift: compounding errors in autoregressive rollouts that severely degrade generation quality over long time horizons. Existing work attempts to address this issue by implicitly leveraging the inherent trade-off between short-time accuracy and long-time consistency through hyperparameter tuning. In this work, we introduce a unifying mathematical framework that makes this tradeoff explicit, formalizing and generalizing hyperparameter-based strategies in existing approaches. Within this framework, we propose a robust, hyperparameter-free model implemented as a conditional diffusion model that balances short-time fidelity with long-time consistency by construction. Our model, Self-refining Neural Surrogate model (SNS), can be implemented as a standalone model that refines its own autoregressive outputs or as a complementary model to existing neural surrogates to ensure long-time consistency. We also demonstrate the numerical feasibility of SNS through high-fidelity simulations of complex dynamical systems over arbitrarily long time horizons.