# Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions

arXiv: https://arxiv.org/abs/2206.04129 · HTML: https://mlsanity.com/paper/2206.04129/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 8 Jun 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps](https://mlsanity.com/paper/2003.06754/) (2020)
- [SuMa++: Efficient LiDAR-based Semantic SLAM](https://mlsanity.com/paper/2105.11320/) (2021)
- [PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences](https://mlsanity.com/paper/2205.13713/) (2022)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- [Weakly Supervised Learning of Rigid 3D Scene Flow](https://mlsanity.com/paper/2102.08945/) (2021)
- [Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks](https://mlsanity.com/paper/2110.04076/) (2021)
- [Self-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion](https://mlsanity.com/paper/2009.10467/) (2020)
- [PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing](https://mlsanity.com/paper/1910.08287/) (2019)
- [Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation](https://mlsanity.com/paper/2201.04501/) (2022)
- [LiDAR-based Recurrent 3D Semantic Segmentation with Temporal Memory Alignment](https://mlsanity.com/paper/2103.02263/) (2021)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [DeepTemporalSeg: Temporally Consistent Semantic Segmentation of 3D LiDAR Scans](https://mlsanity.com/paper/1906.06962/) (2019)
- [Double-Prong ConvLSTM for Spatiotemporal Occupancy Prediction in Dynamic Environments](https://mlsanity.com/paper/2011.09045/) (2020)
- [Self-Supervised Learning of Lidar Segmentation for Autonomous Indoor Navigation](https://mlsanity.com/paper/2012.05897/) (2020)
- [RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecasting](https://mlsanity.com/paper/2005.10863/) (2020)

## Built on by (most cited first, 30 of 35)

- [Domain generalization of 3D semantic segmentation in autonomous driving](https://mlsanity.com/paper/2212.04245/) (2022)
- [MF-MOS: A Motion-Focused Model for Moving Object Segmentation](https://mlsanity.com/paper/2401.17023/) (2024)
- [OpenGraph: Open-Vocabulary Hierarchical 3D Graph Representation in Large-Scale Outdoor Environments](https://mlsanity.com/paper/2403.09412/) (2024)
- [InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data](https://mlsanity.com/paper/2303.03909/) (2023)
- [SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2308.13323/) (2023)
- [MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space Model](https://mlsanity.com/paper/2404.12794/) (2024)
- [DUFOMap: Efficient Dynamic Awareness Mapping](https://mlsanity.com/paper/2403.01449/) (2024)
- [MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features](https://mlsanity.com/paper/2305.07336/) (2023)
- [Radar Instance Transformer: Reliable Moving Instance Segmentation in Sparse Radar Point Clouds](https://mlsanity.com/paper/2309.16435/) (2023)
- [Unsupervised 4D LiDAR Moving Object Segmentation in Stationary Settings with Multivariate Occupancy Time Series](https://mlsanity.com/paper/2212.14750/) (2022)
- [A Dynamic Points Removal Benchmark in Point Cloud Maps](https://mlsanity.com/paper/2307.07260/) (2023)
- [3D LiDAR Mapping in Dynamic Environments Using a 4D Implicit Neural Representation](https://mlsanity.com/paper/2405.03388/) (2024)
- [Self-Supervised Sparse Sensor Fusion for Long Range Perception](https://mlsanity.com/paper/2508.13995/) (2025)
- [BeautyMap: Binary-Encoded Adaptable Ground Matrix for Dynamic Points Removal in Global Maps](https://mlsanity.com/paper/2405.07283/) (2024)
- [SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud](https://mlsanity.com/paper/2406.16279/) (2024)
- [HeLiMOS: A Dataset for Moving Object Segmentation in 3D Point Clouds From Heterogeneous LiDAR Sensors](https://mlsanity.com/paper/2408.06328/) (2024)
- [TruckDrive: Long-Range Autonomous Highway Driving Dataset](https://mlsanity.com/paper/2603.02413/) (2026)
- [Instantaneous Perception of Moving Objects in 3D](https://mlsanity.com/paper/2405.02781/) (2024)
- [StreamMOS: Streaming Moving Object Segmentation with Multi-View Perception and Dual-Span Memory](https://mlsanity.com/paper/2407.17905/) (2024)
- [LiDAR-BEVMTN: Real-Time LiDAR Bird's-Eye View Multi-Task Perception Network for Autonomous Driving](https://mlsanity.com/paper/2307.08850/) (2023)
- [TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perception](https://mlsanity.com/paper/2412.03054/) (2024)
- [Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds](https://mlsanity.com/paper/2507.03441/) (2025)
- [Radar Velocity Transformer: Single-scan Moving Object Segmentation in Noisy Radar Point Clouds](https://mlsanity.com/paper/2507.03463/) (2025)
- [SemRaFiner: Panoptic Segmentation in Sparse and Noisy Radar Point Clouds](https://mlsanity.com/paper/2507.06906/) (2025)
- [MV-MOS: Multi-View Feature Fusion for 3D Moving Object Segmentation](https://mlsanity.com/paper/2408.10602/) (2024)
- [CV-MOS: A Cross-View Model for Motion Segmentation](https://mlsanity.com/paper/2408.13790/) (2024)
- [UniLiPs: Unified LiDAR Pseudo-Labeling with Geometry-Grounded Dynamic Scene Decomposition](https://mlsanity.com/paper/2601.05105/) (2026)
- [Moving Object Segmentation in Point Cloud Data using Hidden Markov Models](https://mlsanity.com/paper/2410.18638/) (2024)
- [HIF: Height Interval Filtering for Efficient Dynamic Points Removal](https://mlsanity.com/paper/2503.06863/) (2025)
- [Temporal Overlapping Prediction: A Self-supervised Pre-training Method for LiDAR Moving Object Segmentation](https://mlsanity.com/paper/2503.07167/) (2025)

## Abstract (arXiv)

A key challenge for autonomous vehicles is to navigate in unseen dynamic environments. Separating moving objects from static ones is essential for navigation, pose estimation, and understanding how other traffic participants are likely to move in the near future. In this work, we tackle the problem of distinguishing 3D LiDAR points that belong to currently moving objects, like walking pedestrians or driving cars, from points that are obtained from non-moving objects, like walls but also parked cars. Our approach takes a sequence of observed LiDAR scans and turns them into a voxelized sparse 4D point cloud. We apply computationally efficient sparse 4D convolutions to jointly extract spatial and temporal features and predict moving object confidence scores for all points in the sequence. We develop a receding horizon strategy that allows us to predict moving objects online and to refine predictions on the go based on new observations. We use a binary Bayes filter to recursively integrate new predictions of a scan resulting in more robust estimation. We evaluate our approach on the SemanticKITTI moving object segmentation challenge and show more accurate predictions than existing methods. Since our approach only operates on the geometric information of point clouds over time, it generalizes well to new, unseen environments, which we evaluate on the Apollo dataset.