mlsanityData to 7 Oct 2026

Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates

arXiv 2412.13074 ↗First posted 17 Dec 2024cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Neural surrogates for partial differential equations (PDEs) have become popular due to their potential to quickly simulate physics. With a few exceptions, neural surrogates generally treat the forward evolution of time-dependent PDEs as a black box by directly predicting the next state. While this is a natural and easy framework for applying neural surrogates, it can be an over-simplified and rigid framework for predicting physics. In this work, we evaluate an alternate framework in which neural solvers predict the temporal derivative and an ODE integrator forwards the solution in time, which has little overhead and is broadly applicable across model architectures and PDEs. We find that by simply changing the training target and introducing numerical integration during inference, neural surrogates can gain accuracy and stability in finely-discretized regimes. Predicting temporal derivatives also allows models to not be constrained to a specific temporal discretization, allowing for flexible time-stepping during inference or training on higher-resolution PDE data. Lastly, we investigate why this framework can be beneficial and in what situations does it work well.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 57 of its 73 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

57 of 73 references.

References · 57

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2016Wide Residual Networkscited by 3,057
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  7. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  8. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  9. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  10. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  11. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  12. 2020Neural Operator: Graph Kernel Network for Partial Differential Equationscited by 722
38 more
  1. 2019Hamiltonian Neural Networkscited by 702
  2. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  3. 2021Machine learning accelerated computational fluid dynamicscited by 461
  4. 2021Neural Operator: Learning Maps Between Function Spacescited by 413
  5. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  6. 2020Lagrangian Neural Networkscited by 388
  7. 2020Multipole Graph Neural Operator for Parametric Partial Differential Equationscited by 343
  8. 2021Choose a Transformer: Fourier or Galerkincited by 333
  9. 2024Transolver: A Fast Transformer Solver for PDEs on General Geometriescited by 299
  10. 2023Geometry-Informed Neural Operator for Large-Scale 3D PDEscited by 293
  11. 2022Transformer for Partial Differential Equations' Operator Learningcited by 270
  12. 2022GraphCast: Learning skillful medium-range global weather forecastingcited by 261
  13. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  14. 2022U-NO: U-shaped Neural Operatorscited by 219
  15. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 205
  16. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solverscited by 205
  17. 2020Machine Learning Force Fieldscited by 204
  18. 2019Towards Physics-informed Deep Learning for Turbulent Flow Predictioncited by 203
  19. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 190
  20. 2024Poseidon: Efficient Foundation Models for PDEscited by 186
  21. 2023GenCast: Diffusion-based ensemble forecasting for medium-range weathercited by 169
  22. 2018Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flowscited by 147
  23. 2024DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Trainingcited by 137
  24. 2024The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learningcited by 134
  25. 2023Scalable Transformer for PDE Surrogate Modelingcited by 123
  26. 2019Hamiltonian Graph Networks with ODE Integratorscited by 107
  27. 2021Learned Coarse Models for Efficient Turbulence Simulationcited by 90
  28. 2024Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equationscited by 87
  29. 2022Lie Point Symmetry Data Augmentation for Neural PDE Solverscited by 76
  30. 2023Multi-Grid Tensorized Fourier Neural Operator for High-Resolution PDEscited by 66
  31. 2023Benchmarking Autoregressive Conditional Diffusion Models for Turbulent Flow Simulationcited by 65
  32. 2024APEBench: A Benchmark for Autoregressive Neural Emulators of PDEscited by 51
  33. 2022Learning to correct spectral methods for simulating turbulent flowscited by 49
  34. 2023From Zero to Turbulence: Generative Modeling for 3D Flow Simulationcited by 46
  35. 2021Learning Dissipative Dynamics in Chaotic Systemscited by 41
  36. 2024Integrating Neural Operators with Diffusion Models Improves Spectral Representation in Turbulence Modelingcited by 36
  37. 2021Graph Neural Networks Accelerated Molecular Dynamicscited by 33
  38. 2020Solving inverse-PDE problems with physics-aware neural networkscited by 27

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025Generative Latent Neural PDE Solver using Flow Matchingcited by 28
  2. 2025Active operator learning with predictive uncertainty quantification for partial differential equationscited by 12
  3. 2025Reframing Generative Models for Physical Systems using Stochastic Interpolantscited by 2
  4. 2026Learning Physical Operators using Neural Operatorscited by 1
  5. 2025Hamiltonian Neural PDE Solvers through Functional Approximationcited by 0
  6. 2025BlastOFormer: Attention and Neural Operator Deep Learning Methods for Explosive Blast Predictioncited by 0
  7. 2025Weight-Parameterization in Continuous Time Deep Neural Networks for Surrogate Modelingcited by 0
  8. 2026The High Explosives and Affected Targets (HEAT) Datasetcited by 0