# 3D Object Detection with a Self-supervised Lidar Scene Flow Backbone

arXiv: https://arxiv.org/abs/2205.00705 · HTML: https://mlsanity.com/paper/2205.00705/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 67 references that are in the map.
- First posted on arXiv 2 May 2022.

## Builds on (most cited first, 30 of 51)

- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [PointPainting: Sequential Fusion for 3D Object Detection](https://mlsanity.com/paper/1911.10150/) (2019)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Learning Correspondence from the Cycle-Consistency of Time](https://mlsanity.com/paper/1903.07593/) (2019)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts](https://mlsanity.com/paper/2012.09165/) (2020)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency](https://mlsanity.com/paper/1809.01649/) (2018)
- [Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases](https://mlsanity.com/paper/2007.13916/) (2020)
- [Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Camera](https://mlsanity.com/paper/1907.05820/) (2019)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)

## Built on by (most cited first, 19 of 19)

- [Vision Language Models in Autonomous Driving: A Survey and Outlook](https://mlsanity.com/paper/2310.14414/) (2023)
- [GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Clouds](https://mlsanity.com/paper/2212.03010/) (2022)
- [Masked Autoencoder for Self-Supervised Pre-training on Lidar Point Clouds](https://mlsanity.com/paper/2207.00531/) (2022)
- [ICP-Flow: LiDAR Scene Flow Estimation with ICP](https://mlsanity.com/paper/2402.17351/) (2024)
- [LidarCLIP or: How I Learned to Talk to Point Clouds](https://mlsanity.com/paper/2212.06858/) (2022)
- [I Can't Believe It's Not Scene Flow!](https://mlsanity.com/paper/2403.04739/) (2024)
- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling](https://mlsanity.com/paper/2402.18146/) (2024)
- [Neural Eulerian Scene Flow Fields](https://mlsanity.com/paper/2410.02031/) (2024)
- [ZeroFlow: Scalable Scene Flow via Distillation](https://mlsanity.com/paper/2305.10424/) (2023)
- [VoteFlow: Enforcing Local Rigidity in Self-Supervised Scene Flow](https://mlsanity.com/paper/2503.22328/) (2025)
- [Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering](https://mlsanity.com/paper/2404.08363/) (2024)
- [GraphRelate3D: Context-Dependent 3D Object Detection with Inter-Object Relationship Graphs](https://mlsanity.com/paper/2405.06782/) (2024)
- [TARS: Traffic-Aware Radar Scene Flow Estimation](https://mlsanity.com/paper/2503.10210/) (2025)
- [MetAdv: A Unified and Interactive Adversarial Testing Platform for Autonomous Driving](https://mlsanity.com/paper/2508.06534/) (2025)
- [RaLiFlow: Scene Flow Estimation with 4D Radar and LiDAR Point Clouds](https://mlsanity.com/paper/2512.10376/) (2025)
- [Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior](https://mlsanity.com/paper/2310.11284/) (2023)
- [Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency](https://mlsanity.com/paper/2312.08879/) (2023)
- [Leveraging Anchor-based LiDAR 3D Object Detection via Point Assisted Sample Selection](https://mlsanity.com/paper/2403.01978/) (2024)

## Abstract (arXiv)

State-of-the-art lidar-based 3D object detection methods rely on supervised learning and large labeled datasets. However, annotating lidar data is resource-consuming, and depending only on supervised learning limits the applicability of trained models. Self-supervised training strategies can alleviate these issues by learning a general point cloud backbone model for downstream 3D vision tasks. Against this backdrop, we show the relationship between self-supervised multi-frame flow representations and single-frame 3D detection hypotheses. Our main contribution leverages learned flow and motion representations and combines a self-supervised backbone with a supervised 3D detection head. First, a self-supervised scene flow estimation model is trained with cycle consistency. Then, the point cloud encoder of this model is used as the backbone of a single-frame 3D object detection head model. This second 3D object detection model learns to utilize motion representations to distinguish dynamic objects exhibiting different movement patterns. Experiments on KITTI and nuScenes benchmarks show that the proposed self-supervised pre-training increases 3D detection performance significantly. https://github.com/emecercelik/ssl-3d-detection.git