mlsanityData to 7 Oct 2026

PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics

arXiv 2409.09811 ↗First posted 15 Sep 2024cs.LG cs.NA math.NAcited by 33

Not on the map. Why?

Abstract (arXiv)

We propose PROSE-FD, a zero-shot multimodal PDE foundational model for simultaneous prediction of heterogeneous two-dimensional physical systems related to distinct fluid dynamics settings. These systems include shallow water equations and the Navier-Stokes equations with incompressible and compressible flow, regular and complex geometries, and different buoyancy settings. This work presents a new transformer-based multi-operator learning approach that fuses symbolic information to perform operator-based data prediction, i.e. non-autoregressive. By incorporating multiple modalities in the inputs, the PDE foundation model builds in a pathway for including mathematical descriptions of the physical behavior. We pre-train our foundation model on 6 parametric families of equations collected from 13 datasets, including over 60K trajectories. Our model outperforms popular operator learning, computer vision, and multi-physics models, in benchmark forward prediction tasks. We test our architecture choices with ablation studies.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 15 Sep 2024.

Builds on

39 of 50 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
27 more
  1. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  4. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  5. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  6. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
  7. 2022FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operatorscited by 748
  8. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  9. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  10. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  11. 2022Fourier Neural Operator with Learned Deformations for PDEs on General Geometriescited by 416
  12. 2022Multimodal Learning with Transformers: A Surveycited by 399
  13. 2022PDEBENCH: An Extensive Benchmark for Scientific Machine Learningcited by 396
  14. 2019Axial Attention in Multidimensional Transformerscited by 350
  15. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 237
  16. 2022End-to-end symbolic regression with transformerscited by 195
  17. 2022Towards Multi-spatiotemporal-scale Generalized PDE Modelingcited by 190
  18. 2016Dual Attention Networks for Multimodal Reasoning and Matchingcited by 113
  19. 2023In-Context Operator Learning with Data Prompts for Differential Equation Problemscited by 98
  20. 2023Multiple Physics Pretraining for Physical Surrogate Modelscited by 90
  21. 2022Deep Symbolic Regression for Recurrent Sequencescited by 53
  22. 2024Towards a Foundation Model for Partial Differential Equations: Multi-Operator Learning and Extrapolationcited by 45
  23. 2019NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Datacited by 42
  24. 2024PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Lawscited by 30
  25. 2023PROSE: Predicting Operators and Symbolic Expressions using Multimodal Transformerscited by 16
  26. 2024LeMON: Learning to Learn Multi-Operator Networkscited by 13
  27. 2023Finite Expression Methods for Discovering Physical Laws from Datacited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 14; 2026: 15.

33 papers.

Sort:

By area:

  1. 2024VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Predictioncited by 22
  2. 2025BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamicscited by 16
  3. 2024Neural Scaling Laws of Deep ReLU and Deep Operator Network: A Theoretical Studycited by 13
  4. 2025MORPH: PDE Foundation Models with Arbitrary Data Modalitycited by 12
  5. 2024DeepONet as a Multi-Operator Extrapolation Model: Distributed Pretraining with Physics-Informed Fine-Tuningcited by 11
  6. 2025A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptionscited by 8
  7. 2024Time-Series Forecasting, Knowledge Distillation, and Refinement within a Multimodal PDE Foundation Modelcited by 6
  8. 2025Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transitioncited by 6
  9. 2025A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theorycited by 6
  10. 2025Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Modelscited by 5
  11. 2026CompNO: A Novel Foundation Model approach for solving Partial Differential Equationscited by 4
  12. 2026OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inferencecited by 4
21 more
  1. 2025Cauchy Random Features for Operator Learning in Sobolev Spacecited by 3
  2. 2025Continuum Transformers Perform In-Context Learning by Operator Gradient Descentcited by 3
  3. 2025Flow marching for a generative PDE foundation modelcited by 3
  4. 2026Graph In-Context Operator Networks for Generalizable Spatiotemporal Predictioncited by 3
  5. 2026Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulencecited by 3
  6. 2025Convolutional-neural-operator-based transfer learning for solving PDEscited by 2
  7. 2026Latent Generative Solvers for Generalizable Long-Term Physics Simulationcited by 2
  8. 2025OmniField: Conditioned Neural Fields for Robust Multimodal Spatiotemporal Learningcited by 1
  9. 2026MVNN: A Measure-Valued Neural Network for Learning McKean-Vlasov Dynamics from Particle Datacited by 1
  10. 2026Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networkscited by 1
  11. 2026A Multimodal Vision Transformer-based Modeling Framework for Prediction of Fluid Flows in Energy Systemscited by 1
  12. 2026Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learningcited by 1
  13. 2025SPUS: A Lightweight and Parameter-Efficient Foundation Model for PDEscited by 0
  14. 2025FMint-SDE: A Multimodal Foundation Model for Accelerating Numerical Simulation of SDEs via Error Correctioncited by 0
  15. 2025In-Context Multi-Operator Learning with DeepOSetscited by 0
  16. 2026Compositional Neural Operators for Multi-Dimensional Fluid Dynamicscited by 0
  17. 2026Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospectscited by 0
  18. 2026Kastor: An efficient fine-tuning strategy for generative emulation of PDE simulationscited by 0
  19. 2026Distillation of Foundation Models for Time-dependent PDEscited by 0
  20. 2026Learnable composition for neural operatorscited by 0
  21. 2026Does Text Steer Neural PDE Surrogates? A Controlled Diagnostic with OperatorCLIPcited by 0