mlsanityData to 7 Oct 2026

Inferring Fluid Dynamics via Inverse Rendering

arXiv 2304.04446 ↗First posted 10 Apr 2023cs.CV cs.GRcited by 3

Not on the map. Why?

Abstract (arXiv)

Humans have a strong intuitive understanding of physical processes such as fluid falling by just a glimpse of such a scene picture, i.e., quickly derived from our immersive visual experiences in memory. This work achieves such a photo-to-fluid-dynamics reconstruction functionality learned from unannotated videos, without any supervision of ground-truth fluid dynamics. In a nutshell, a differentiable Euler simulator modeled with a ConvNet-based pressure projection solver, is integrated with a volumetric renderer, supporting end-to-end/coherent differentiable dynamic simulation and rendering. By endowing each sampled point with a fluid volume value, we derive a NeRF-like differentiable renderer dedicated from fluid data; and thanks to this volume-augmented representation, fluid dynamics could be inversely inferred from the error signal between the rendered result and ground-truth video frame (i.e., inverse rendering). Experiments on our generated Fluid Fall datasets and DPI Dam Break dataset are conducted to demonstrate both effectiveness and generalization ability of our method.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 85 references that are in the map. First posted on arXiv 10 Apr 2023.

Builds on

43 of 85 references.

References · 43

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  4. 2020Neural Sparse Voxel Fieldscited by 959
  5. 2020Learning to Simulate Complex Physics with Graph Networkscited by 867
  6. 2020Learning Mesh-Based Simulation with Graph Networkscited by 840
  7. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 503
  8. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  9. 2017Neural 3D Mesh Renderercited by 437
  10. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  11. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  12. 2020Neural Scene Graphs for Dynamic Scenescited by 298
31 more
  1. 2019Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Datacited by 261
  2. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  3. 2018Learning Particle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluidscited by 219
  4. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  5. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  6. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 152
  7. 2018Deep Learning Methods for Reynolds-Averaged Navier-Stokes Simulations of Airfoil Flowscited by 147
  8. 2016Accelerating Eulerian Fluid Simulation With Convolutional Networkscited by 128
  9. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  10. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  11. 2018Deep Fluids: A Generative Network for Parameterized Fluid Simulationscited by 121
  12. 2016A Compositional Object-Based Approach to Learning Physical Dynamicscited by 119
  13. 20213D Neural Scene Representations for Visuomotor Controlcited by 110
  14. 2021Unsupervised Discovery of Object Radiance Fieldscited by 100
  15. 2018ChainQueen: A Real-Time Differentiable Physical Simulator for Soft Roboticscited by 98
  16. 2018Latent-space Physics: Towards Learning the Temporal Evolution of Fluid Flowcited by 97
  17. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  18. 2016Unsupervised Learning of 3D Structure from Imagescited by 67
  19. 2016Learning Physical Intuition of Block Towers by Examplecited by 66
  20. 2019Inverse Path Tracing for Joint Material and Lighting Estimationcited by 62
  21. 2015Learning Visual Predictive Models of Physics for Playing Billiardscited by 58
  22. 2020Visual Grounding of Learned Physical Modelscited by 54
  23. 2022Physics Informed Neural Fields for Smoke Reconstruction with Sparse Datacited by 52
  24. 2021Spline-PINN: Approaching PDEs without Data using Fast, Physics-Informed Hermite-Spline CNNscited by 38
  25. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 37
  26. 2022NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fieldscited by 36
  27. 2022Eikonal Fields for Refractive Novel-View Synthesiscited by 30
  28. 20183D-Aware Scene Manipulation via Inverse Graphicscited by 24
  29. 2016"What happens if..." Learning to Predict the Effect of Forces in Imagescited by 23
  30. 2017Learning A Physical Long-term Predictorcited by 5
  31. 2022NeReF: Neural Refractive Field for Fluid Surface Reconstruction and Implicit Representationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 1.

3 papers.

  1. 2023Inferring Hybrid Neural Fluid Fields from Videoscited by 18
  2. 2024DEL: Discrete Element Learner for Learning 3D Particle Dynamics with Neural Renderingcited by 4
  3. 2026Harnessing AI for Inverse Partial Differential Equation Problems: Past, Present, and Prospectscited by 0