mlsanityData to 14 Oct 2026

Learning to Estimate Single-View Volumetric Flow Motions without 3D Supervision

arXiv 2302.14470 ↗First posted 28 Feb 2023cs.CV cs.LG physics.flu-dyncited by 5

Not on the map. Why?

Abstract (arXiv)

We address the challenging problem of jointly inferring the 3D flow and volumetric densities moving in a fluid from a monocular input video with a deep neural network. Despite the complexity of this task, we show that it is possible to train the corresponding networks without requiring any 3D ground truth for training. In the absence of ground truth data we can train our model with observations from real-world capture setups instead of relying on synthetic reconstructions. We make this unsupervised training approach possible by first generating an initial prototype volume which is then moved and transported over time without the need for volumetric supervision. Our approach relies purely on image-based losses, an adversarial discriminator network, and regularization. Our method can estimate long-term sequences in a stable manner, while achieving closely matching targets for inputs such as rising smoke plumes.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 31 of its 65 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

31 of 65 references.

References · 31

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  3. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  4. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  5. 2017Neural 3D Mesh Renderercited by 437
  6. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
  7. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 405
  8. 2018The relativistic discriminator: a key element missing from standard GANcited by 238
  9. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  10. 2019Understanding and Improving Layer Normalizationcited by 217
  11. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  12. 2019SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimizationcited by 159
19 more
  1. 2020Differentiable Rendering: A Surveycited by 123
  2. 2018Deep Fluids: A Generative Network for Parameterized Fluid Simulationscited by 121
  3. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  4. 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
  5. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  6. 2018Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimationcited by 69
  7. 2020Self-Supervised Monocular Scene Flow Estimationcited by 64
  8. 2018SPNets: Differentiable Fluid Dynamics for Deep Neural Networkscited by 62
  9. 2022Physics Informed Neural Fields for Smoke Reconstruction with Sparse Datacited by 52
  10. 2020UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learningcited by 49
  11. 2021Self-Supervised Multi-Frame Monocular Scene Flowcited by 29
  12. 2019Mono-SF: Multi-View Geometry Meets Single-View Depth for Monocular Scene Flow Estimation of Dynamic Traffic Scenescited by 27
  13. 2019PWOC-3D: Deep Occlusion-Aware End-to-End Scene Flow Estimationcited by 20
  14. 2021Global Transport for Fluid Reconstruction with Learned Self-Supervisioncited by 18
  15. 2016Unsupervised convolutional neural networks for motion estimationcited by 12
  16. 2019Learning Residual Flow as Dynamic Motion from Stereo Videoscited by 12
  17. 2020Lagrangian Neural Style Transfer for Fluidscited by 11
  18. 2019Transport-Based Neural Style Transfer for Smoke Simulationscited by 10
  19. 2018Coupled Fluid Density and Motion from Single Viewscited by 5

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 4.

5 papers.

  1. 2025TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videoscited by 11
  2. 2023HelmFluid: Learning Helmholtz Dynamics for Interpretable Fluid Predictioncited by 7
  3. 2025Unraveling Normal Anatomy via Fluid-Driven Anomaly Randomizationcited by 4
  4. 2025Learning an Implicit Physics Model for Image-based Fluid Simulationcited by 3
  5. 2025SmokeSVD: Smoke Reconstruction from A Single View via Progressive Novel View Synthesis and Refinement with Diffusion Modelscited by 0