# VICON: Vision In-Context Operator Networks for Multi-Physics Fluid Dynamics Prediction

arXiv: https://arxiv.org/abs/2411.16063 · HTML: https://mlsanity.com/paper/2411.16063/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 51 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Builds on (most cited first, 30 of 37)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators](https://mlsanity.com/paper/2202.11214/) (2022)
- [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017)
- [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [Geometry-Informed Neural Operator for Large-Scale 3D PDEs](https://mlsanity.com/paper/2309.00583/) (2023)
- [U-FNO -- An enhanced Fourier neural operator-based deep-learning model for multiphase flow](https://mlsanity.com/paper/2109.03697/) (2021)
- [Transformer for Partial Differential Equations' Operator Learning](https://mlsanity.com/paper/2205.13671/) (2022)
- [Sequential Modeling Enables Scalable Learning for Large Vision Models](https://mlsanity.com/paper/2312.00785/) (2023)
- [Poseidon: Efficient Foundation Models for PDEs](https://mlsanity.com/paper/2405.19101/) (2024)
- [Towards Multi-spatiotemporal-scale Generalized PDE Modeling](https://mlsanity.com/paper/2209.15616/) (2022)
- [MIONet: Learning multiple-input operators via tensor product](https://mlsanity.com/paper/2202.06137/) (2022)
- [In-Context Operator Learning with Data Prompts for Differential Equation Problems](https://mlsanity.com/paper/2304.07993/) (2023)
- [Multiple Physics Pretraining for Physical Surrogate Models](https://mlsanity.com/paper/2310.02994/) (2023)
- [Fourier-MIONet: Fourier-enhanced multiple-input neural operators for multiphase modeling of geological carbon sequestration](https://mlsanity.com/paper/2303.04778/) (2023)
- [Towards a Foundation Model for Partial Differential Equations: Multi-Operator Learning and Extrapolation](https://mlsanity.com/paper/2404.12355/) (2024)
- [Efficient Learning of Mesh-Based Physical Simulation with BSMS-GNN](https://mlsanity.com/paper/2210.02573/) (2022)
- [PROSE-FD: A Multimodal PDE Foundation Model for Learning Multiple Operators for Forecasting Fluid Dynamics](https://mlsanity.com/paper/2409.09811/) (2024)
- [Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning](https://mlsanity.com/paper/2402.15734/) (2024)
- [PDE Generalization of In-Context Operator Networks: A Study on 1D Scalar Nonlinear Conservation Laws](https://mlsanity.com/paper/2401.07364/) (2024)
- [Conformalized-DeepONet: A Distribution-Free Framework for Uncertainty Quantification in Deep Operator Networks](https://mlsanity.com/paper/2402.15406/) (2024)

## Built on by (most cited first, 25 of 25)

- [DISCO: learning to DISCover an evolution Operator for multi-physics-agnostic prediction](https://mlsanity.com/paper/2504.19496/) (2025)
- [BCAT: A Block Causal Transformer for PDE Foundation Models for Fluid Dynamics](https://mlsanity.com/paper/2501.18972/) (2025)
- [A Multimodal PDE Foundation Model for Prediction and Scientific Text Descriptions](https://mlsanity.com/paper/2502.06026/) (2025)
- [Test-time Generalization for Physics through Neural Operator Splitting](https://mlsanity.com/paper/2602.00884/) (2026)
- [A Deep Learning Framework for Multi-Operator Learning: Architectures and Approximation Theory](https://mlsanity.com/paper/2510.25379/) (2025)
- [Towards a Physics Foundation Model](https://mlsanity.com/paper/2509.13805/) (2025)
- [Cauchy Random Features for Operator Learning in Sobolev Space](https://mlsanity.com/paper/2503.00300/) (2025)
- [CompNO: A Novel Foundation Model approach for solving Partial Differential Equations](https://mlsanity.com/paper/2601.07384/) (2026)
- [Multiple Neural Operators Achieve Near-Optimal Rates for Multi-Task Learning](https://mlsanity.com/paper/2605.22724/) (2026)
- [Emergent Transfer of a Physics Foundation Model from Simulation to Laboratory Turbulence](https://mlsanity.com/paper/2606.01470/) (2026)
- [Continuum Transformers Perform In-Context Learning by Operator Gradient Descent](https://mlsanity.com/paper/2505.17838/) (2025)
- [Graph In-Context Operator Networks for Generalizable Spatiotemporal Prediction](https://mlsanity.com/paper/2603.12725/) (2026)
- [Generalization Bounds and Statistical Guarantees for Multi-Task and Multiple Operator Learning with MNO Networks](https://mlsanity.com/paper/2604.01961/) (2026)
- [Tadpole: Autoencoders as Foundation Models for 3D PDEs with Online Learning](https://mlsanity.com/paper/2605.15284/) (2026)
- [Chain of Operators: An Inference-Time Harness for In-Context Operator Learning](https://mlsanity.com/paper/2606.12318/) (2026)
- [MVNN: A Measure-Valued Neural Network for Learning McKean-Vlasov Dynamics from Particle Data](https://mlsanity.com/paper/2604.00333/) (2026)
- [A Multimodal Vision Transformer-based Modeling Framework for Prediction of Fluid Flows in Energy Systems](https://mlsanity.com/paper/2604.02483/) (2026)
- [Neural operator discovery from heterogeneous trajectories](https://mlsanity.com/paper/2607.23337/) (2026)
- [TGLF-WINN: Data-Efficient Deep Learning Surrogate for Turbulent Transport Modeling in Fusion](https://mlsanity.com/paper/2509.07024/) (2025)
- [Compositional Neural Operators for Multi-Dimensional Fluid Dynamics](https://mlsanity.com/paper/2605.11691/) (2026)
- [SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models](https://mlsanity.com/paper/2605.17985/) (2026)
- [Operator Learning for Reconstructing Flow Fields from Sparse Measurements: a Language Model Approach](https://mlsanity.com/paper/2605.23712/) (2026)
- [A foundation model of numerical intelligence with cross-disciplinary generalization](https://mlsanity.com/paper/2607.28432/) (2026)
- [Kernel Methods for Learning Operators with Multiple Inputs and Outputs](https://mlsanity.com/paper/2608.11831/) (2026)
- [Learnable composition for neural operators](https://mlsanity.com/paper/2609.03069/) (2026)

## Abstract (arXiv)

In-Context Operator Networks (ICONs) have demonstrated the ability to learn operators across diverse partial differential equations using few-shot, in-context learning. However, existing ICONs process each spatial point as an individual token, severely limiting computational efficiency when handling dense data in higher spatial dimensions. We propose Vision In-Context Operator Networks (VICON), which integrates vision transformer architectures to efficiently process 2D data through patch-wise operations while preserving ICON's adaptability to multiphysics systems and varying timesteps. Evaluated across three fluid dynamics benchmarks, VICON significantly outperforms state-of-the-art baselines: DPOT and MPP, reducing the averaged last-step rollout error by 37.9% compared to DPOT and 44.7% compared to MPP, while requiring only 72.5% and 34.8% of their respective inference times. VICON naturally supports flexible rollout strategies with varying timestep strides, enabling immediate deployment in imperfect measurement systems where sampling frequencies may differ or frames might be dropped - common challenges in real-world settings - without requiring retraining or interpolation. In these realistic scenarios, VICON exhibits remarkable robustness, experiencing only 24.41% relative performance degradation compared to 71.37%-74.49% degradation in baseline methods, demonstrating its versatility for deploying in realistic applications. Our scripts for processing datasets and code are publicly available at https://github.com/Eydcao/VICON.