mlsanityData to 14 Oct 2026

Unsupervised Learning of 3D Scene Flow from Monocular Camera

arXiv 2206.03673 ↗First posted 8 Jun 2022cs.CV cs.ROcited by 7

Not on the map. Why?

Abstract (arXiv)

Scene flow represents the motion of points in the 3D space, which is the counterpart of the optical flow that represents the motion of pixels in the 2D image. However, it is difficult to obtain the ground truth of scene flow in the real scenes, and recent studies are based on synthetic data for training. Therefore, how to train a scene flow network with unsupervised methods based on real-world data shows crucial significance. A novel unsupervised learning method for scene flow is proposed in this paper, which utilizes the images of two consecutive frames taken by monocular camera without the ground truth of scene flow for training. Our method realizes the goal that training scene flow network with real-world data, which bridges the gap between training data and test data and broadens the scope of available data for training. Unsupervised learning of scene flow in this paper mainly consists of two parts: (i) depth estimation and camera pose estimation, and (ii) scene flow estimation based on four different loss functions. Depth estimation and camera pose estimation obtain the depth maps and camera pose between two consecutive frames, which provide further information for the next scene flow estimation. After that, we used depth consistency loss, dynamic-static consistency loss, Chamfer loss, and Laplacian regularization loss to carry out unsupervised training of the scene flow network. To our knowledge, this is the first paper that realizes the unsupervised learning of 3D scene flow from monocular camera. The experiment results on KITTI show that our method for unsupervised learning of scene flow meets great performance compared to traditional methods Iterative Closest Point (ICP) and Fast Global Registration (FGR). The source code is available at: https://github.com/IRMVLab/3DUnMonoFlow.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 27 of its 50 references that are in the map. First posted on arXiv 8 Jun 2022.

Builds on

27 of 50 references.

References · 27

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  4. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  5. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  6. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  7. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  8. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  9. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  10. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  11. 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
  12. 2019SelFlow: Self-Supervised Learning of Optical Flowcited by 128
15 more
  1. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 109
  2. 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
  3. 2018Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimationcited by 69
  4. 2020Self-Supervised Monocular Scene Flow Estimationcited by 64
  5. 2019DDFlow: Learning Optical Flow with Unlabeled Data Distillationcited by 63
  6. 2019Deep Rigid Instance Scene Flowcited by 54
  7. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  8. 2019SENSE: a Shared Encoder Network for Scene-flow Estimationcited by 38
  9. 2020Hierarchical Attention Learning of Scene Flow in 3D Point Cloudscited by 35
  10. 2020PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimizationcited by 35
  11. 2020Unsupervised Learning of Depth, Optical Flow and Pose with Occlusion from 3D Geometrycited by 21
  12. 2019LiDAR-Flow: Dense Scene Flow Estimation from Sparse LiDAR and Stereo Imagescited by 12
  13. 2020Spherical Interpolated Convolutional Network with Distance-Feature Density for 3D Semantic Segmentation of Point Cloudscited by 6
  14. 2021Unsupervised Learning of Monocular Depth and Ego-Motion Using Multiple Maskscited by 6
  15. 2016A Continuous Optimization Approach for Efficient and Accurate Scene Flowcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 1.

7 papers.

  1. 2023RegFormer: An Efficient Projection-Aware Transformer Network for Large-Scale Point Cloud Registrationcited by 37
  2. 2022What Matters for 3D Scene Flow Networkcited by 31
  3. 2023DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Cloudscited by 14
  4. 2023RLSAC: Reinforcement Learning enhanced Sample Consensus for End-to-End Robust Estimationcited by 3
  5. 2022Pseudo-LiDAR for Visual Odometrycited by 2
  6. 2023LTS-NET: End-to-end Unsupervised Learning of Long-Term 3D Stable objectscited by 1
  7. 2024Long-Term 3D Point Tracking By Cost Volume Fusioncited by 0