# Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion

arXiv: https://arxiv.org/abs/2009.10467 · HTML: https://mlsanity.com/paper/2009.10467/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2020.

## Built on by (most cited first, 30 of 41)

- [Neural Scene Flow Prior](https://mlsanity.com/paper/2111.01253/) (2021)
- [Weakly Supervised Learning of Rigid 3D Scene Flow](https://mlsanity.com/paper/2102.08945/) (2021)
- [MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronization](https://mlsanity.com/paper/2101.06605/) (2021)
- [Fast Neural Scene Flow](https://mlsanity.com/paper/2304.09121/) (2023)
- [SeFlow: A Self-Supervised Scene Flow Method in Autonomous Driving](https://mlsanity.com/paper/2407.01702/) (2024)
- [Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimation](https://mlsanity.com/paper/2207.07522/) (2022)
- [Hidden Gems: 4D Radar Scene Flow Learning Using Cross-Modal Supervision](https://mlsanity.com/paper/2303.00462/) (2023)
- [Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions](https://mlsanity.com/paper/2206.04129/) (2022)
- [ICP-Flow: LiDAR Scene Flow Estimation with ICP](https://mlsanity.com/paper/2402.17351/) (2024)
- [Self-Supervised Scene Flow Estimation with 4-D Automotive Radar](https://mlsanity.com/paper/2203.01137/) (2022)
- [SCOOP: Self-Supervised Correspondence and Optimization-Based Scene Flow](https://mlsanity.com/paper/2211.14020/) (2022)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds](https://mlsanity.com/paper/2210.04458/) (2022)
- [Self-Supervised 3D Scene Flow Estimation Guided by Superpoints](https://mlsanity.com/paper/2305.02528/) (2023)
- [SCTN: Sparse Convolution-Transformer Network for Scene Flow Estimation](https://mlsanity.com/paper/2105.04447/) (2021)
- [I Can't Believe It's Not Scene Flow!](https://mlsanity.com/paper/2403.04739/) (2024)
- [Zero-Shot Monocular Scene Flow Estimation in the Wild](https://mlsanity.com/paper/2501.10357/) (2025)
- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [Re-Evaluating LiDAR Scene Flow for Autonomous Driving](https://mlsanity.com/paper/2304.02150/) (2023)
- [3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling](https://mlsanity.com/paper/2402.18146/) (2024)
- [Neural Eulerian Scene Flow Fields](https://mlsanity.com/paper/2410.02031/) (2024)
- [ZeroFlow: Scalable Scene Flow via Distillation](https://mlsanity.com/paper/2305.10424/) (2023)
- [Multi-Body Neural Scene Flow](https://mlsanity.com/paper/2310.10301/) (2023)
- [Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/2104.04724/) (2021)
- [Self-Supervised Robust Scene Flow Estimation via the Alignment of Probability Density Functions](https://mlsanity.com/paper/2203.12193/) (2022)
- [RCP: Recurrent Closest Point for Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/2205.11028/) (2022)
- [Self-Supervised Bird's Eye View Motion Prediction with Cross-Modality Signals](https://mlsanity.com/paper/2401.11499/) (2024)
- [Dynamic Semantic Occupancy Mapping using 3D Scene Flow and Closed-Form Bayesian Inference](https://mlsanity.com/paper/2108.03180/) (2021)
- [Monocular Road Planar Parallax Estimation](https://mlsanity.com/paper/2111.11089/) (2021)
- [Learning Scene Flow in 3D Point Clouds with Noisy Pseudo Labels](https://mlsanity.com/paper/2203.12655/) (2022)

## Abstract (arXiv)

Most of the current scene flow methods choose to model scene flow as a per point translation vector without differentiating between static and dynamic components of 3D motion. In this work we present an alternative method for end-to-end scene flow learning by joint estimation of non-rigid residual flow and ego-motion flow for dynamic 3D scenes. We propose to learn the relative rigid transformation from a pair of point clouds followed by an iterative refinement. We then learn the non-rigid flow from transformed inputs with the deducted rigid part of the flow. Furthermore, we extend the supervised framework with self-supervisory signals based on the temporal consistency property of a point cloud sequence. Our solution allows both training in a supervised mode complemented by self-supervisory loss terms as well as training in a fully self-supervised mode. We demonstrate that decomposition of scene flow into non-rigid flow and ego-motion flow along with an introduction of the self-supervisory signals allowed us to outperform the current state-of-the-art supervised methods.