# Learning Vortex Dynamics for Fluid Inference and Prediction

arXiv: https://arxiv.org/abs/2301.11494 · HTML: https://mlsanity.com/paper/2301.11494/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 52 references that are in the map.
- First posted on arXiv 27 Jan 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Learning to Simulate Complex Physics with Graph Networks](https://mlsanity.com/paper/2002.09405/) (2020)
- [Learning Mesh-Based Simulation with Graph Networks](https://mlsanity.com/paper/2010.03409/) (2020)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [DiffTaichi: Differentiable Programming for Physical Simulation](https://mlsanity.com/paper/1910.00935/) (2019)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)
- [Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids](https://mlsanity.com/paper/1810.01566/) (2018)
- [Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1708.00588/) (2017)
- [Machine Learning of Linear Differential Equations using Gaussian Processes](https://mlsanity.com/paper/1701.02440/) (2017)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [Accelerating Eulerian Fluid Simulation With Convolutional Networks](https://mlsanity.com/paper/1607.03597/) (2016)
- [A Compositional Object-Based Approach to Learning Physical Dynamics](https://mlsanity.com/paper/1612.00341/) (2016)
- [Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flow](https://mlsanity.com/paper/1802.10123/) (2018)
- [Visual Interaction Networks](https://mlsanity.com/paper/1706.01433/) (2017)
- [Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data](https://mlsanity.com/paper/2206.06577/) (2022)
- [tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow](https://mlsanity.com/paper/1801.09710/) (2018)
- [NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields](https://mlsanity.com/paper/2203.01762/) (2022)
- [Lagrangian Neural Style Transfer for Fluids](https://mlsanity.com/paper/2005.00803/) (2020)
- [Neural Vortex Method: from Finite Lagrangian Particles to Infinite Dimensional Eulerian Dynamics](https://mlsanity.com/paper/2006.04178/) (2020)
- [Learning to Estimate and Refine Fluid Motion with Physical Dynamics](https://mlsanity.com/paper/2206.10480/) (2022)

## Built on by (most cited first, 17 of 17)

- [Reconstruction and Simulation of Elastic Objects with Spring-Mass 3D Gaussians](https://mlsanity.com/paper/2403.09434/) (2024)
- [Physics-Informed Computer Vision: A Review and Perspectives](https://mlsanity.com/paper/2305.18035/) (2023)
- [NVFi: Neural Velocity Fields for 3D Physics Learning from Dynamic Videos](https://mlsanity.com/paper/2312.06398/) (2023)
- [Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks](https://mlsanity.com/paper/2401.11611/) (2024)
- [Inferring Hybrid Neural Fluid Fields from Videos](https://mlsanity.com/paper/2312.06561/) (2023)
- [Fluid Simulation on Neural Flow Maps](https://mlsanity.com/paper/2312.14635/) (2023)
- [QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer](https://mlsanity.com/paper/2404.07988/) (2024)
- [TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos](https://mlsanity.com/paper/2508.09811/) (2025)
- [HelmFluid: Learning Helmholtz Dynamics for Interpretable Fluid Prediction](https://mlsanity.com/paper/2310.10565/) (2023)
- [PGODE: Towards High-quality System Dynamics Modeling](https://mlsanity.com/paper/2311.06554/) (2023)
- [Representing Flow Fields with Divergence-Free Kernels for Reconstruction](https://mlsanity.com/paper/2504.01913/) (2025)
- [Data-Efficient Inference of Neural Fluid Fields via SciML Foundation Model](https://mlsanity.com/paper/2412.13897/) (2024)
- [PhysGaia: A Physics-Aware Benchmark with Multi-Body Interactions for Dynamic Novel View Synthesis](https://mlsanity.com/paper/2506.02794/) (2025)
- [DeepLag: Discovering Deep Lagrangian Dynamics for Intuitive Fluid Prediction](https://mlsanity.com/paper/2402.02425/) (2024)
- [PINP: Physics-Informed Neural Predictor with latent estimation of fluid flows](https://mlsanity.com/paper/2504.06070/) (2025)
- [WildSmoke: Ready-to-Use Dynamic 3D Smoke Assets from a Single Video in the Wild](https://mlsanity.com/paper/2509.11114/) (2025)
- [Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospects](https://mlsanity.com/paper/2605.16966/) (2026)

## Abstract (arXiv)

We propose a novel differentiable vortex particle (DVP) method to infer and predict fluid dynamics from a single video. Lying at its core is a particle-based latent space to encapsulate the hidden, Lagrangian vortical evolution underpinning the observable, Eulerian flow phenomena. Our differentiable vortex particles are coupled with a learnable, vortex-to-velocity dynamics mapping to effectively capture the complex flow features in a physically-constrained, low-dimensional space. This representation facilitates the learning of a fluid simulator tailored to the input video that can deliver robust, long-term future predictions. The value of our method is twofold: first, our learned simulator enables the inference of hidden physics quantities (e.g., velocity field) purely from visual observation; secondly, it also supports future prediction, constructing the input video's sequel along with its future dynamics evolution. We compare our method with a range of existing methods on both synthetic and real-world videos, demonstrating improved reconstruction quality, visual plausibility, and physical integrity.