# Unsupervised 4D LiDAR Moving Object Segmentation in Stationary Settings with Multivariate Occupancy Time Series

arXiv: https://arxiv.org/abs/2212.14750 · HTML: https://mlsanity.com/paper/2212.14750/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 30 Dec 2022.

## Builds on (most cited first, 28 of 28)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021)
- [Unsupervised Scalable Representation Learning for Multivariate Time Series](https://mlsanity.com/paper/1901.10738/) (2019)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Unsupervised Representation Learning for Time Series with Temporal Neighborhood Coding](https://mlsanity.com/paper/2106.00750/) (2021)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering](https://mlsanity.com/paper/2103.17070/) (2021)
- [Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation](https://mlsanity.com/paper/2008.01550/) (2020)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- [Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions](https://mlsanity.com/paper/2206.04129/) (2022)
- [Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation](https://mlsanity.com/paper/2201.04501/) (2022)
- [Unsupervised Domain Adaptation for LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2109.15286/) (2021)
- [A Random Finite Set Approach for Dynamic Occupancy Grid Maps with Real-Time Application](https://mlsanity.com/paper/1605.02406/) (2016)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [Dynamic Occupancy Grid Mapping with Recurrent Neural Networks](https://mlsanity.com/paper/2011.08659/) (2020)
- [How important are faces for person re-identification?](https://mlsanity.com/paper/2010.06307/) (2020)
- [Robust Moving Objects Detection in Lidar Data Exploiting Visual Cues](https://mlsanity.com/paper/1609.09267/) (2016)
- [Motion Estimation in Occupancy Grid Maps in Stationary Settings Using Recurrent Neural Networks](https://mlsanity.com/paper/1909.11387/) (2019)
- [Deep Object Tracking on Dynamic Occupancy Grid Maps Using RNNs](https://mlsanity.com/paper/1805.08986/) (2018)
- [Semantics-Guided Moving Object Segmentation with 3D LiDAR](https://mlsanity.com/paper/2205.03186/) (2022)

## Built on by (most cited first, 8 of 8)

- [InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data](https://mlsanity.com/paper/2303.03909/) (2023)
- [MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features](https://mlsanity.com/paper/2305.07336/) (2023)
- [Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Clouds](https://mlsanity.com/paper/2309.16435/) (2023)
- [SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud](https://mlsanity.com/paper/2406.16279/) (2024)
- [LiOn-XA: Unsupervised Domain Adaptation via LiDAR-Only Cross-Modal Adversarial Training](https://mlsanity.com/paper/2410.15833/) (2024)
- [StreamMOS: Streaming Moving Object Segmentation with Multi-View Perception and Dual-Span Memory](https://mlsanity.com/paper/2407.17905/) (2024)
- [CV-MOS: A Cross-View Model for Motion Segmentation](https://mlsanity.com/paper/2408.13790/) (2024)
- [Temporal Overlapping Prediction: A Self-supervised Pre-training Method for LiDAR Moving Object Segmentation](https://mlsanity.com/paper/2503.07167/) (2025)

## Abstract (arXiv)

In this work, we address the problem of unsupervised moving object segmentation (MOS) in 4D LiDAR data recorded from a stationary sensor, where no ground truth annotations are involved. Deep learning-based state-of-the-art methods for LiDAR MOS strongly depend on annotated ground truth data, which is expensive to obtain and scarce in existence. To close this gap in the stationary setting, we propose a novel 4D LiDAR representation based on multivariate time series that relaxes the problem of unsupervised MOS to a time series clustering problem. More specifically, we propose modeling the change in occupancy of a voxel by a multivariate occupancy time series (MOTS), which captures spatio-temporal occupancy changes on the voxel level and its surrounding neighborhood. To perform unsupervised MOS, we train a neural network in a self-supervised manner to encode MOTS into voxel-level feature representations, which can be partitioned by a clustering algorithm into moving or stationary. Experiments on stationary scenes from the Raw KITTI dataset show that our fully unsupervised approach achieves performance that is comparable to that of supervised state-of-the-art approaches.