# A Multimodal Vision Transformer-based Modeling Framework for Prediction of Fluid Flows in Energy Systems

arXiv: https://arxiv.org/abs/2604.02483 · HTML: https://mlsanity.com/paper/2604.02483/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 25 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 21 of 21)

- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [Neural Operator: Graph Kernel Network for Partial Differential Equations](https://mlsanity.com/paper/2003.03485/) (2020)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [ClimaX: A foundation model for weather and climate](https://mlsanity.com/paper/2301.10343/) (2023)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior](https://mlsanity.com/paper/2306.00258/) (2023)
- [Scaling transformer neural networks for skillful and reliable medium-range weather forecasting](https://mlsanity.com/paper/2312.03876/) (2023)
- [Multiple Physics Pretraining for Physical Surrogate Models](https://mlsanity.com/paper/2310.02994/) (2023)
- [PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics](https://mlsanity.com/paper/2409.09811/) (2024)
- [Differentiability in Unrolled Training of Neural Physics Simulators on Transient Dynamics](https://mlsanity.com/paper/2402.12971/) (2024)
- [VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction](https://mlsanity.com/paper/2411.16063/) (2024)
- [PhysiX: A Foundation Model for Physics Simulations](https://mlsanity.com/paper/2506.17774/) (2025)
- [OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time Scales](https://mlsanity.com/paper/2510.18707/) (2025)
- [AERIS: Argonne Earth Systems Model for Reliable and Skillful Predictions](https://mlsanity.com/paper/2509.13523/) (2025)
- [EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale](https://mlsanity.com/paper/2510.24173/) (2025)
- [Autoregressive Renaissance in Neural PDE Solvers](https://mlsanity.com/paper/2310.19763/) (2023)

## Built on by (most cited first, 1 of 1)

- [Prediction of Viscoelastic Droplet Impact Dynamics Using a Vision Transformer-Based Approach](https://mlsanity.com/paper/2606.23940/) (2026)

## Abstract (arXiv)

Computational fluid dynamics (CFD) simulations of complex fluid flows in energy systems are prohibitively expensive due to strong nonlinearities and multiscale-multiphysics interactions. In this work, we present a transformer-based modeling framework for prediction of fluid flows, and demonstrate it for high-pressure gas injection phenomena relevant to reciprocating engines. The approach employs a hierarchical Vision Transformer (SwinV2-UNet) architecture that processes multimodal flow datasets from multi-fidelity simulations. The model architecture is conditioned on auxiliary tokens explicitly encoding the data modality and time increment. Model performance is assessed on two different tasks: (1) spatiotemporal rollouts, where the model autoregressively predicts the flow state at future times; and (2) feature transformation, where the model infers unobserved fields/views from observed fields/views. We train separate models on multimodal datasets generated from in-house CFD simulations of argon jet injection into a nitrogen environment, encompassing multiple grid resolutions, turbulence models, and equations of state. The resulting data-driven models learn to generalize across resolutions and modalities, accurately forecasting the flow evolution and reconstructing missing flow-field information from limited views. This work demonstrates how large vision transformer-based models can be adapted to advance predictive modeling of complex fluid flow systems.