# A Spatiotemporal Correspondence Approach to Unsupervised LiDAR Segmentation with Traffic Applications

arXiv: https://arxiv.org/abs/2308.12433 · HTML: https://mlsanity.com/paper/2308.12433/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Invariant Information Clustering for Unsupervised Image Classification and Segmentation](https://mlsanity.com/paper/1807.06653/) (2018)
- [SCAN: Learning to Classify Images without Labels](https://mlsanity.com/paper/2005.12320/) (2020)
- [Contrastive Clustering](https://mlsanity.com/paper/2009.09687/) (2020)
- [Understanding self-supervised Learning Dynamics without Contrastive Pairs](https://mlsanity.com/paper/2102.06810/) (2021)
- [SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud](https://mlsanity.com/paper/1710.07368/) (2017)
- [SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud](https://mlsanity.com/paper/1809.08495/) (2018)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering](https://mlsanity.com/paper/2103.17070/) (2021)
- [SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instances](https://mlsanity.com/paper/2002.09147/) (2020)
- [JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields](https://mlsanity.com/paper/1904.00699/) (2019)
- [Autoregressive Unsupervised Image Segmentation](https://mlsanity.com/paper/2007.08247/) (2020)
- [SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/1909.08291/) (2019)
- [Multi-scale Interaction for Real-time LiDAR Data Segmentation on an Embedded Platform](https://mlsanity.com/paper/2008.09162/) (2020)
- [Patchwork: Concentric Zone-based Region-wise Ground Segmentation with Ground Likelihood Estimation Using a 3D LiDAR Sensor](https://mlsanity.com/paper/2108.05560/) (2021)
- [Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation](https://mlsanity.com/paper/2201.04501/) (2022)
- [Towards Demystifying Representation Learning with Non-contrastive Self-supervision](https://mlsanity.com/paper/2110.04947/) (2021)
- [Semantic Segmentation of 3D LiDAR Data in Dynamic Scene Using Semi-supervised Learning](https://mlsanity.com/paper/1809.00426/) (2018)
- [Rethinking 3D LiDAR Point Cloud Segmentation](https://mlsanity.com/paper/2008.03928/) (2020)
- [Dynamic Object Aware LiDAR SLAM based on Automatic Generation of Training Data](https://mlsanity.com/paper/2104.03657/) (2021)
- [An Efficient Semi-Automated Scheme for Infrastructure LiDAR Annotation](https://mlsanity.com/paper/2301.10732/) (2023)

## Abstract (arXiv)

We address the problem of unsupervised semantic segmentation of outdoor LiDAR point clouds in diverse traffic scenarios. The key idea is to leverage the spatiotemporal nature of a dynamic point cloud sequence and introduce drastically stronger augmentation by establishing spatiotemporal correspondences across multiple frames. We dovetail clustering and pseudo-label learning in this work. Essentially, we alternate between clustering points into semantic groups and optimizing models using point-wise pseudo-spatiotemporal labels with a simple learning objective. Therefore, our method can learn discriminative features in an unsupervised learning fashion. We show promising segmentation performance on Semantic-KITTI, SemanticPOSS, and FLORIDA benchmark datasets covering scenarios in autonomous vehicle and intersection infrastructure, which is competitive when compared against many existing fully supervised learning methods. This general framework can lead to a unified representation learning approach for LiDAR point clouds incorporating domain knowledge.