mlsanityData to 7 Oct 2026

Weakly Supervised Learning of Rigid 3D Scene Flow

arXiv 2102.08945 ↗First posted 17 Feb 2021cs.CV cs.AI cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

We propose a data-driven scene flow estimation algorithm exploiting the observation that many 3D scenes can be explained by a collection of agents moving as rigid bodies. At the core of our method lies a deep architecture able to reason at the object-level by considering 3D scene flow in conjunction with other 3D tasks. This object level abstraction, enables us to relax the requirement for dense scene flow supervision with simpler binary background segmentation mask and ego-motion annotations. Our mild supervision requirements make our method well suited for recently released massive data collections for autonomous driving, which do not contain dense scene flow annotations. As output, our model provides low-level cues like pointwise flow and higher-level cues such as holistic scene understanding at the level of rigid objects. We further propose a test-time optimization refining the predicted rigid scene flow. We showcase the effectiveness and generalization capacity of our method on four different autonomous driving datasets. We release our source code and pre-trained models under urlgithub.com/zgojcic/Rigid3DSceneFlow.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 47 of its 98 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

47 of 98 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2017Mask R-CNNcited by 7,259
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  7. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  8. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  9. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  10. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  11. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  12. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
35 more
  1. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  2. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  3. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 318
  4. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  5. 2020RPM-Net: Robust Point Matching using Learned Featurescited by 181
  6. 2017SEGCloud: Semantic Segmentation of 3D Point Cloudscited by 178
  7. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 168
  8. 2021Deep Parametric Continuous Convolutional Neural Networkscited by 161
  9. 2018SPLATNet: Sparse Lattice Networks for Point Cloud Processingcited by 158
  10. 2019Modeling Point Clouds with Self-Attention and Gumbel Subset Samplingcited by 151
  11. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 145
  12. 2020FLOT: Scene Flow on Point Clouds Guided by Optimal Transportcited by 129
  13. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  14. 2020Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Cloudscited by 116
  15. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 108
  16. 2020Robust Learning Through Cross-Task Consistencycited by 105
  17. 2019Just Go with the Flow: Self-Supervised Scene Flow Estimationcited by 98
  18. 2018MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objectscited by 95
  19. 2016VolumeDeform: Real-time Volumetric Non-rigid Reconstructioncited by 75
  20. 2018Deep Part Induction from Articulated Object Pairscited by 66
  21. 2019FlowNet3D++: Geometric Losses For Deep Scene Flow Estimationcited by 55
  22. 2019Deep Rigid Instance Scene Flowcited by 54
  23. 2020CaSPR: Learning Canonical Spatiotemporal Point Cloud Representationscited by 51
  24. 2016SE3-Nets: Learning Rigid Body Motion using Deep Neural Networkscited by 48
  25. 2020Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimationcited by 43
  26. 2020Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motioncited by 41
  27. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  28. 2019SENSE: a Shared Encoder Network for Scene-flow Estimationcited by 38
  29. 2019PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Cloudscited by 38
  30. 2019PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processingcited by 32
  31. 2019EM-Fusion: Dynamic Object-Level SLAM with Probabilistic Data Associationcited by 28
  32. 20206D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inferencecited by 23
  33. 2018Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMCcited by 19
  34. 2020Synchronizing Probability Measures on Rotations via Optimal Transportcited by 19
  35. 2017Multiframe Scene Flow with Piecewise Rigid Motioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 15; 2023: 18; 2024: 11; 2025: 6; 2026: 2.

60 papers.

Sort:

By area:

  1. 2021Lepard: Learning partial point cloud matching in rigid and deformable scenescited by 85
  2. 2021Intrinsic Dimension, Persistent Homology and Generalization in Neural Networkscited by 70
  3. 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
  4. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  5. 2023Fast Neural Scene Flowcited by 39
  6. 2023Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervisioncited by 36
  7. 2022Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutionscited by 35
  8. 2024ICP-Flow: LiDAR Scene Flow Estimation with ICPcited by 34
  9. 2021Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronizationcited by 30
  10. 2022Self-Supervised Scene Flow Estimation with 4-D Automotive Radarcited by 29
  11. 2022Deformation and Correspondence Aware Unsupervised Synthetic-to-Real Scene Flow Estimation for Point Cloudscited by 28
  12. 2022SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flowcited by 28
38 more
  1. 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Videocited by 27
  2. 2022No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfacescited by 25
  3. 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 24
  4. 2023Self-Supervised 3D Scene Flow Estimation Guided by Superpointscited by 24
  5. 2021SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimationcited by 23
  6. 2023Dynamic LiDAR Re-simulation using Compositional Neural Fieldscited by 22
  7. 2024I Can't Believe It's Not Scene Flow!cited by 21
  8. 2023milliFlow: Scene Flow Estimation on mmWave Radar Point Cloud for Human Motion Sensingcited by 20
  9. 2025Zero-Shot Monocular Scene Flow Estimation in the Wildcited by 20
  10. 2023Re-Evaluating LiDAR Scene Flow for Autonomous Drivingcited by 17
  11. 2021Residual 3D Scene Flow Learning with Context-Aware Feature Extractioncited by 15
  12. 2023Living Scenes: Multi-object Relocalization and Reconstruction in Changing 3D Environmentscited by 15
  13. 2023ZeroFlow: Scalable Scene Flow via Distillationcited by 13
  14. 2023Multi-Body Neural Scene Flowcited by 12
  15. 2022Unsupervised confidence for LiDAR depth maps and applicationscited by 10
  16. 2023Deep Transfer Learning for Intelligent Vehicle Perception: a Surveycited by 10
  17. 2021Learning Scene Dynamics from Point Cloud Sequencescited by 9
  18. 2023Variational Inference for SDEs Driven by Fractional Noisecited by 9
  19. 2022Self-Supervised Robust Scene Flow Estimation via the Alignment of Probability Density Functionscited by 7
  20. 2021VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidarscited by 6
  21. 2022Robust Self-Supervised LiDAR Odometry via Representative Structure Discovery and 3D Inherent Error Modelingcited by 6
  22. 2023Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detectioncited by 6
  23. 2024DiffSF: Diffusion Models for Scene Flow Estimationcited by 6
  24. 2024Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signalscited by 5
  25. 2021Sensor-Guided Optical Flowcited by 4
  26. 2022Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labelscited by 3
  27. 2024Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clusteringcited by 3
  28. 2024SMORE: Simultaneous Map and Object REconstructioncited by 3
  29. 2025TARS: Traffic-Aware Radar Scene Flow Estimationcited by 3
  30. 20224DAC: Learning Attribute Compression for Dynamic Point Cloudscited by 2
  31. 2022Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistancecited by 2
  32. 2023Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Priorcited by 2
  33. 2025SemanticFlow: A Self-Supervised Framework for Joint Scene Flow Prediction and Instance Segmentation in Dynamic Environmentscited by 2
  34. 2025Weakly and Self-Supervised Class-Agnostic Motion Prediction for Autonomous Drivingcited by 2
  35. 2023PointFlowHop: Green and Interpretable Scene Flow Estimation from Consecutive Point Cloudscited by 1
  36. 2023Exploiting Implicit Rigidity Constraints via Weight-Sharing Aggregation for Scene Flow Estimation from Point Cloudscited by 1
  37. 2023DEFLOW: Self-supervised 3D Motion Estimation of Debris Flowcited by 1
  38. 2023Aligning Bird-Eye View Representation of Point Cloud Sequences using Scene Flowcited by 1

50 most cited of 60 All 60 (JSON)