# Adversarial Self-Supervised Scene Flow Estimation

arXiv: https://arxiv.org/abs/2011.00551 · HTML: https://mlsanity.com/paper/2011.00551/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Self-Supervised Scene Flow Estimation with 4-D Automotive Radar](https://mlsanity.com/paper/2203.01137/) (2022)
- [Self-Supervised 3D Scene Flow Estimation Guided by Superpoints](https://mlsanity.com/paper/2305.02528/) (2023)
- [Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Clouds](https://mlsanity.com/paper/2104.04724/) (2021)
- [Learning Scene Dynamics from Point Cloud Sequences](https://mlsanity.com/paper/2111.08755/) (2021)
- [Self-Supervised Robust Scene Flow Estimation via the Alignment of Probability Density Functions](https://mlsanity.com/paper/2203.12193/) (2022)
- [Self-SuperFlow: Self-supervised Scene Flow Prediction in Stereo Sequences](https://mlsanity.com/paper/2206.15296/) (2022)

## Abstract (arXiv)

This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, eg for recognizing actions, or avoiding collisions. Training a neural network via supervised learning for scene flow is impractical, as this requires manual annotations for each 3D point at each new timestamp for each scene. To that end, we seek for a self-supervised approach, where a network learns a latent metric to distinguish between points translated by flow estimations and the target point cloud. Our adversarial metric learning includes a multi-scale triplet loss on sequences of two-point clouds as well as a cycle consistency loss. Furthermore, we outline a benchmark for self-supervised scene flow estimation: the Scene Flow Sandbox. The benchmark consists of five datasets designed to study individual aspects of flow estimation in progressive order of complexity, from a moving object to real-world scenes. Experimental evaluation on the benchmark shows that our approach obtains state-of-the-art self-supervised scene flow results, outperforming recent neighbor-based approaches. We use our proposed benchmark to expose shortcomings and draw insights on various training setups. We find that our setup captures motion coherence and preserves local geometries. Dealing with occlusions, on the other hand, is still an open challenge.