# Deep Rigid Instance Scene Flow

arXiv: https://arxiv.org/abs/1904.08913 · HTML: https://mlsanity.com/paper/1904.08913/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2019.

## Built on by (most cited first, 30 of 54)

- [SpatialTracker: Tracking Any 2D Pixels in 3D Space](https://mlsanity.com/paper/2404.04319/) (2024)
- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [TAPVid-3D: A Benchmark for Tracking Any Point in 3D](https://mlsanity.com/paper/2407.05921/) (2024)
- [Weakly Supervised Learning of Rigid 3D Scene Flow](https://mlsanity.com/paper/2102.08945/) (2021)
- [CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2111.10502/) (2021)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization](https://mlsanity.com/paper/2101.06605/) (2021)
- [IFOR: Iterative Flow Minimization for Robotic Object Rearrangement](https://mlsanity.com/paper/2202.00732/) (2022)
- [Tangent Space Backpropagation for 3D Transformation Groups](https://mlsanity.com/paper/2103.12032/) (2021)
- [Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk](https://mlsanity.com/paper/2105.08248/) (2021)
- [HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embedding](https://mlsanity.com/paper/2105.07751/) (2021)
- [What Matters for 3D Scene Flow Network](https://mlsanity.com/paper/2207.09143/) (2022)
- [Self-Supervised Multi-Frame Monocular Scene Flow](https://mlsanity.com/paper/2105.02216/) (2021)
- [Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video](https://mlsanity.com/paper/2503.21761/) (2025)
- [SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow](https://mlsanity.com/paper/2211.14020/) (2022)
- [OcclusionFusion: Occlusion-aware Motion Estimation for Real-time Dynamic 3D Reconstruction](https://mlsanity.com/paper/2203.07977/) (2022)
- [Temporally Consistent Online Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2111.09337/) (2021)
- [RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation](https://mlsanity.com/paper/2309.15082/) (2023)
- [SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation](https://mlsanity.com/paper/2105.04447/) (2021)
- [Zero-Shot Monocular Scene Flow Estimation in the Wild](https://mlsanity.com/paper/2501.10357/) (2025)
- [Binary TTC: A Temporal Geofence for Autonomous Navigation](https://mlsanity.com/paper/2101.04777/) (2021)
- [Re-Evaluating LiDAR Scene Flow for Autonomous Driving](https://mlsanity.com/paper/2304.02150/) (2023)
- [Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow](https://mlsanity.com/paper/2403.07432/) (2024)
- [M-FUSE: Multi-frame Fusion for Scene Flow Estimation](https://mlsanity.com/paper/2207.05704/) (2022)
- [GMSF: Global Matching Scene Flow](https://mlsanity.com/paper/2305.17432/) (2023)
- [DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds](https://mlsanity.com/paper/2308.04383/) (2023)
- [Object-centric 3D Motion Field for Robot Learning from Human Videos](https://mlsanity.com/paper/2506.04227/) (2025)
- [Multi-Body Neural Scene Flow](https://mlsanity.com/paper/2310.10301/) (2023)
- [Deep Feedback Inverse Problem Solver](https://mlsanity.com/paper/2101.07719/) (2021)
- [Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusion](https://mlsanity.com/paper/2303.12017/) (2023)

## Abstract (arXiv)

In this paper we tackle the problem of scene flow estimation in the context of self-driving. We leverage deep learning techniques as well as strong priors as in our application domain the motion of the scene can be composed by the motion of the robot and the 3D motion of the actors in the scene. We formulate the problem as energy minimization in a deep structured model, which can be solved efficiently in the GPU by unrolling a Gaussian-Newton solver. Our experiments in the challenging KITTI scene flow dataset show that we outperform the state-of-the-art by a very large margin, while being 800 times faster.