mlsanityData to 7 Oct 2026

Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance

arXiv 2209.04945 ↗First posted 11 Sep 2022cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Scene flow represents the 3D motion of each point in the scene, which explicitly describes the distance and the direction of each point's movement. Scene flow estimation is used in various applications such as autonomous driving fields, activity recognition, and virtual reality fields. As it is challenging to annotate scene flow with ground truth for real-world data, this leaves no real-world dataset available to provide a large amount of data with ground truth for scene flow estimation. Therefore, many works use synthesized data to pre-train their network and real-world LiDAR data to finetune. Unlike the previous unsupervised learning of scene flow in point clouds, we propose to use odometry information to assist the unsupervised learning of scene flow and use real-world LiDAR data to train our network. Supervised odometry provides more accurate shared cost volume for scene flow. In addition, the proposed network has mask-weighted warp layers to get a more accurate predicted point cloud. The warp operation means applying an estimated pose transformation or scene flow to a source point cloud to obtain a predicted point cloud and is the key to refining scene flow from coarse to fine. When performing warp operations, the points in different states use different weights for the pose transformation and scene flow transformation. We classify the states of points as static, dynamic, and occluded, where the static masks are used to divide static and dynamic points, and the occlusion masks are used to divide occluded points. The mask-weighted warp layer indicates that static masks and occlusion masks are used as weights when performing warp operations. Our designs are proved to be effective in ablation experiments. The experiment results show the promising prospect of an odometry-assisted unsupervised learning method for 3D scene flow in real-world data.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 11 Sep 2022.

Builds on

29 of 42 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  5. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  6. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  7. 2020FLOT: Scene Flow on Point Clouds Guided by Optimal Transportcited by 129
  8. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  9. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 108
  10. 2019Just Go with the Flow: Self-Supervised Scene Flow Estimationcited by 98
  11. 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
  12. 2019LO-Net: Deep Real-time Lidar Odometrycited by 59
17 more
  1. 2019Deep Rigid Instance Scene Flowcited by 54
  2. 2020Scene Flow from Point Clouds with or without Learningcited by 44
  3. 2020Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motioncited by 41
  4. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  5. 2020Hierarchical Attention Learning of Scene Flow in 3D Point Cloudscited by 35
  6. 2020PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimizationcited by 34
  7. 2022What Matters for 3D Scene Flow Networkcited by 31
  8. 2022Deformation and Correspondence Aware Unsupervised Synthetic-to-Real Scene Flow Estimation for Point Cloudscited by 28
  9. 2021Efficient 3D Deep LiDAR Odometrycited by 25
  10. 2020PointTrackNet: An End-to-End Network For 3-D Object Detection and Tracking From Point Cloudscited by 20
  11. 2020LodoNet: A Deep Neural Network with 2D Keypoint Matchingfor 3D LiDAR Odometry Estimationcited by 19
  12. 2019Learning Independent Object Motion from Unlabelled Stereoscopic Videoscited by 17
  13. 2021Residual 3D Scene Flow Learning with Context-Aware Feature Extractioncited by 15
  14. 2019DeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Networkcited by 14
  15. 2017CNN for IMU Assisted Odometry Estimation using Velodyne LiDARcited by 11
  16. 2021Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Cloudscited by 11
  17. 2022Interactive Multi-scale Fusion of 2D and 3D Features for Multi-object Trackingcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolationcited by 3
  2. 2024EgoFlowNet: Non-Rigid Scene Flow from Point Clouds with Ego-Motion Supportcited by 0