# Scalable Scene Flow from Point Clouds in the Real World

arXiv: https://arxiv.org/abs/2103.01306 · HTML: https://mlsanity.com/paper/2103.01306/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation](https://mlsanity.com/paper/1502.02734/) (2015)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/1910.06528/) (2019)
- [Deep Parametric Continuous Convolutional Neural Networks](https://mlsanity.com/paper/2101.06742/) (2021)
- [IntentNet: Learning to Predict Intention from Raw Sensor Data](https://mlsanity.com/paper/2101.07907/) (2021)
- [Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping](https://mlsanity.com/paper/1709.07857/) (2017)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Clouds](https://mlsanity.com/paper/1906.05332/) (2019)
- [MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps](https://mlsanity.com/paper/2003.06754/) (2020)
- [Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling](https://mlsanity.com/paper/1902.08295/) (2019)
- [FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation](https://mlsanity.com/paper/1912.01438/) (2019)
- [StarNet: Targeted Computation for Object Detection in Point Clouds](https://mlsanity.com/paper/1908.11069/) (2019)
- [Scene Flow from Point Clouds with or without Learning](https://mlsanity.com/paper/2011.00320/) (2020)
- [PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/1911.12408/) (2019)
- [Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation](https://mlsanity.com/paper/2005.10266/) (2020)
- [High precision grasp pose detection in dense clutter](https://mlsanity.com/paper/1603.01564/) (2016)
- [PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing](https://mlsanity.com/paper/1910.08287/) (2019)

## Built on by (most cited first, 30 of 50)

- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [Dynamic 3D Scene Analysis by Point Cloud Accumulation](https://mlsanity.com/paper/2207.12394/) (2022)
- [Fast Neural Scene Flow](https://mlsanity.com/paper/2304.09121/) (2023)
- [SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving](https://mlsanity.com/paper/2407.01702/) (2024)
- [Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision](https://mlsanity.com/paper/2303.00462/) (2023)
- [ICP-Flow: LiDAR Scene Flow Estimation with ICP](https://mlsanity.com/paper/2402.17351/) (2024)
- [DeFlow: Decoder of Scene Flow Network in Autonomous Driving](https://mlsanity.com/paper/2401.16122/) (2024)
- [Deformation and Correspondence Aware Unsupervised Synthetic-to-Real Scene Flow Estimation for Point Clouds](https://mlsanity.com/paper/2203.16895/) (2022)
- [Real-Time Optical Flow for Vehicular Perception with Low- and High-Resolution Event Cameras](https://mlsanity.com/paper/2112.10591/) (2021)
- [Self-Supervised 3D Scene Flow Estimation Guided by Superpoints](https://mlsanity.com/paper/2305.02528/) (2023)
- [Flow4D: Leveraging 4D Voxel Network for LiDAR Scene Flow Estimation](https://mlsanity.com/paper/2407.07995/) (2024)
- [I Can't Believe It's Not Scene Flow!](https://mlsanity.com/paper/2403.04739/) (2024)
- [Zero-Shot Monocular Scene Flow Estimation in the Wild](https://mlsanity.com/paper/2501.10357/) (2025)
- [RaTrack: Moving Object Detection and Tracking with 4D Radar Point Cloud](https://mlsanity.com/paper/2309.09737/) (2023)
- [milliFlow: Scene Flow Estimation on mmWave Radar Point Cloud for Human Motion Sensing](https://mlsanity.com/paper/2306.17010/) (2023)
- [4D Unsupervised Object Discovery](https://mlsanity.com/paper/2210.04801/) (2022)
- [Re-Evaluating LiDAR Scene Flow for Autonomous Driving](https://mlsanity.com/paper/2304.02150/) (2023)
- [FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulation](https://mlsanity.com/paper/2505.10075/) (2025)
- [Neural Eulerian Scene Flow Fields](https://mlsanity.com/paper/2410.02031/) (2024)
- [ZeroFlow: Scalable Scene Flow via Distillation](https://mlsanity.com/paper/2305.10424/) (2023)
- [SSF: Sparse Long-Range Scene Flow for Autonomous Driving](https://mlsanity.com/paper/2501.17821/) (2025)
- [Multi-Body Neural Scene Flow](https://mlsanity.com/paper/2310.10301/) (2023)
- [HiMo: High-Speed Objects Motion Compensation in Point Clouds](https://mlsanity.com/paper/2503.00803/) (2025)
- [VoteFlow: Enforcing Local Rigidity in Self-Supervised Scene Flow](https://mlsanity.com/paper/2503.22328/) (2025)
- [FAST3D: Flow-Aware Self-Training for 3D Object Detectors](https://mlsanity.com/paper/2110.09355/) (2021)
- [MambaFlow: A Novel and Flow-guided State Space Model for Scene Flow Estimation](https://mlsanity.com/paper/2502.16907/) (2025)
- [DeltaFlow: An Efficient Multi-frame Scene Flow Estimation Method](https://mlsanity.com/paper/2508.17054/) (2025)
- [Sequential Point Clouds: A Survey](https://mlsanity.com/paper/2204.09337/) (2022)
- [DiffSF: Diffusion Models for Scene Flow Estimation](https://mlsanity.com/paper/2403.05327/) (2024)
- [BeautyMap: Binary-Encoded Adaptable Ground Matrix for Dynamic Points Removal in Global Maps](https://mlsanity.com/paper/2405.07283/) (2024)

## Abstract (arXiv)

Autonomous vehicles operate in highly dynamic environments necessitating an accurate assessment of which aspects of a scene are moving and where they are moving to. A popular approach to 3D motion estimation, termed scene flow, is to employ 3D point cloud data from consecutive LiDAR scans, although such approaches have been limited by the small size of real-world, annotated LiDAR data. In this work, we introduce a new large-scale dataset for scene flow estimation derived from corresponding tracked 3D objects, which is ~1,000× larger than previous real-world datasets in terms of the number of annotated frames. We demonstrate how previous works were bounded based on the amount of real LiDAR data available, suggesting that larger datasets are required to achieve state-of-the-art predictive performance. Furthermore, we show how previous heuristics for operating on point clouds such as down-sampling heavily degrade performance, motivating a new class of models that are tractable on the full point cloud. To address this issue, we introduce the FastFlow3D architecture which provides real time inference on the full point cloud. Additionally, we design human-interpretable metrics that better capture real world aspects by accounting for ego-motion and providing breakdowns per object type. We hope that this dataset may provide new opportunities for developing real world scene flow systems.