Deep Rigid Instance Scene Flow
Not on the map. Why?
Abstract (arXiv)
In this paper we tackle the problem of scene flow estimation in the context of self-driving. We leverage deep learning techniques as well as strong priors as in our application domain the motion of the scene can be composed by the motion of the robot and the 3D motion of the actors in the scene. We formulate the problem as energy minimization in a deep structured model, which can be solved efficiently in the GPU by unrolling a Gaussian-Newton solver. Our experiments in the challenging KITTI scene flow dataset show that we outperform the state-of-the-art by a very large margin, while being 800 times faster.
Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Apr 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 16; 2022: 10; 2023: 12; 2024: 9; 2025: 5; 2026: 2.
54 papers.
Sort:
By area:
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2021Neural Scene Flow Priorcited by 91
- 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
- 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
- 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
- 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
- 2021Learning to Segment Rigid Motions from Two Framescited by 45
- 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
- 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
- 2021Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walkcited by 39
- 2021HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embeddingcited by 36
- 2022What Matters for 3D Scene Flow Networkcited by 31
38 more
- 2021Self-Supervised Multi-Frame Monocular Scene Flowcited by 29
- 2022SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flowcited by 28
- 2022OcclusionFusion: Occlusion-aware Motion Estimation for Real-time Dynamic 3D Reconstructioncited by 27
- 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Videocited by 27
- 2021Temporally Consistent Online Depth Estimation in Dynamic Scenescited by 26
- 2023RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimationcited by 26
- 2021SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimationcited by 23
- 2021Binary TTC: A Temporal Geofence for Autonomous Navigationcited by 20
- 2025Zero-Shot Monocular Scene Flow Estimation in the Wildcited by 20
- 2023Re-Evaluating LiDAR Scene Flow for Autonomous Drivingcited by 17
- 2024Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flowcited by 15
- 2022M-FUSE: Multi-frame Fusion for Scene Flow Estimationcited by 14
- 2023GMSF: Global Matching Scene Flowcited by 14
- 2023DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Cloudscited by 14
- 2025Object-centric 3D Motion Field for Robot Learning from Human Videoscited by 13
- 2023Multi-Body Neural Scene Flowcited by 12
- 2021Deep Feedback Inverse Problem Solvercited by 11
- 2023Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusioncited by 10
- 2023Recurrent Generic Contour-based Instance Segmentation with Progressive Learningcited by 9
- 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planningcited by 9
- 2022Unsupervised Learning of 3D Scene Flow from Monocular Cameracited by 7
- 2024Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Cloudscited by 7
- 2021Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structurecited by 6
- 2022SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selectioncited by 6
- 2024DiffSF: Diffusion Models for Scene Flow Estimationcited by 6
- 2023A Quadratic Synchronization Rule for Distributed Deep Learningcited by 5
- 2025DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modelingcited by 5
- 2023EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigiditycited by 4
- 2022Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labelscited by 3
- 2024Self-Supervised Multi-Frame Neural Scene Flowcited by 3
- 2021MonoPLFlowNet: Permutohedral Lattice FlowNet for Real-Scale 3D Scene FlowEstimation with Monocular Imagescited by 2
- 2022Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistancecited by 2
- 2023Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Priorcited by 2
- 2024Segment Anything in Light Fields for Real-Time Applications via Constrained Promptingcited by 2
- 2025SplitGaussian: Reconstructing Dynamic Scenes via Visual Geometry Decompositioncited by 2
- 2022Depth Estimation from Single-shot Monocular Endoscope Image Using Image Domain Adaptation And Edge-Aware Depth Estimationcited by 1
- 2023Edge-aware Consistent Stereo Video Depth Estimationcited by 1
- 2024RMS-FlowNet++: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Cloudscited by 1
50 most cited of 54 All 54 (JSON)
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2026PCSTracker: Long-Term Scene Flow Estimation for Point Cloud Sequences20 Mar 2026
- 2025DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modeling2 Dec 2025
- 2025SplitGaussian: Reconstructing Dynamic Scenes via Visual Geometry Decomposition6 Aug 2025
- 2025Object-centric 3D Motion Field for Robot Learning from Human Videos4 Jun 2025
- 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video27 Mar 2025
- 2025Zero-Shot Monocular Scene Flow Estimation in the Wild17 Jan 2025
- 2024Segment Anything in Light Fields for Real-Time Applications via Constrained Prompting21 Nov 2024
- 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3D8 Jul 2024
- 2024EgoFlowNet: Non-Rigid Scene Flow from Point Clouds with Ego-Motion Support3 Jul 2024
- 2024RMS-FlowNet++: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds1 Jul 2024
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Space5 Apr 2024
38 more
- 2024Self-Supervised Multi-Frame Neural Scene Flow24 Mar 2024
- 2024Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow12 Mar 2024
- 2024DiffSF: Diffusion Models for Scene Flow Estimation8 Mar 2024
- 2024Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Clouds16 Feb 2024
- 2023A Quadratic Synchronization Rule for Distributed Deep Learning22 Oct 2023
- 2023Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior17 Oct 2023
- 2023Multi-Body Neural Scene Flow16 Oct 2023
- 2023RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation26 Sep 2023
- 2023EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity4 Sep 2023
- 2023DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Clouds8 Aug 2023
- 2023Milestones in Autonomous Driving and Intelligent Vehicles Part II: Perception and Planning3 Jun 2023
- 2023GMSF: Global Matching Scene Flow27 May 2023
- 2023Edge-aware Consistent Stereo Video Depth Estimation4 May 2023
- 2023Re-Evaluating LiDAR Scene Flow for Autonomous Driving4 Apr 2023
- 2023Learning Optical Flow and Scene Flow with Bidirectional Camera-LiDAR Fusion21 Mar 2023
- 2023Recurrent Generic Contour-based Instance Segmentation with Progressive Learning21 Jan 2023
- 2022SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow25 Nov 2022
- 2022SF2SE3: Clustering Scene Flow into SE(3)-Motions via Proposal and Selection18 Sep 2022
- 2022Unsupervised Learning of 3D Scene Flow with 3D Odometry Assistance11 Sep 2022
- 2022What Matters for 3D Scene Flow Network19 Jul 2022
- 2022M-FUSE: Multi-frame Fusion for Scene Flow Estimation12 Jul 2022
- 2022Unsupervised Learning of 3D Scene Flow from Monocular Camera8 Jun 2022
- 2022Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels23 Mar 2022
- 2022OcclusionFusion: Occlusion-aware Motion Estimation for Real-time Dynamic 3D Reconstruction15 Mar 2022
- 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangement1 Feb 2022
- 2022Depth Estimation from Single-shot Monocular Endoscope Image Using Image Domain Adaptation And Edge-Aware Depth Estimation12 Jan 2022
- 2021MonoPLFlowNet: Permutohedral Lattice FlowNet for Real-Scale 3D Scene FlowEstimation with Monocular Images24 Nov 2021
- 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation20 Nov 2021
- 2021Temporally Consistent Online Depth Estimation in Dynamic Scenes17 Nov 2021
- 2021Neural Scene Flow Prior1 Nov 2021
- 2021Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structure16 Jun 2021
- 2021Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walk18 May 2021
- 2021HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embedding17 May 2021
- 2021SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation10 May 2021
- 2021Self-Supervised Multi-Frame Monocular Scene Flow5 May 2021
- 2021Tangent Space Backpropagation for 3D Transformation Groups22 Mar 2021
- 2021Accurate Visual-Inertial SLAM by Feature Re-identification26 Feb 2021
- 2021Weakly Supervised Learning of Rigid 3D Scene Flow17 Feb 2021
50 newest of 54 All 54 (JSON)
- 2024DiffSF: Diffusion Models for Scene Flow Estimationcited by 6
- 2024DiffSF: Diffusion Models for Scene Flow Estimation8 Mar 2024
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2021Neural Scene Flow Priorcited by 91
- 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3Dcited by 69
- 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
- 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
- 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
- 2021Learning to Segment Rigid Motions from Two Framescited by 45
- 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
- 2021Tangent Space Backpropagation for 3D Transformation Groupscited by 40
- 2021Self-Point-Flow: Self-Supervised Scene Flow Estimation from Point Clouds with Optimal Transport and Random Walkcited by 39
- 2021HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embeddingcited by 36
- 2022What Matters for 3D Scene Flow Networkcited by 31
12 most cited of 52 All 52 (JSON)
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2026PCSTracker: Long-Term Scene Flow Estimation for Point Cloud Sequences20 Mar 2026
- 2025SplitGaussian: Reconstructing Dynamic Scenes via Visual Geometry Decomposition6 Aug 2025
- 2025Object-centric 3D Motion Field for Robot Learning from Human Videos4 Jun 2025
- 2025Uni4D: Unifying Visual Foundation Models for 4D Modeling from a Single Video27 Mar 2025
- 2025Zero-Shot Monocular Scene Flow Estimation in the Wild17 Jan 2025
- 2024Segment Anything in Light Fields for Real-Time Applications via Constrained Prompting21 Nov 2024
- 2024TAPVid-3D: A Benchmark for Tracking Any Point in 3D8 Jul 2024
- 2024EgoFlowNet: Non-Rigid Scene Flow from Point Clouds with Ego-Motion Support3 Jul 2024
- 2024RMS-FlowNet++: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds1 Jul 2024
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Space5 Apr 2024
- 2024Self-Supervised Multi-Frame Neural Scene Flow24 Mar 2024
12 newest of 52 All 52 (JSON)