# SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud

arXiv: https://arxiv.org/abs/2406.16279 · HTML: https://mlsanity.com/paper/2406.16279/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [Spherical Transformer for LiDAR-based 3D Recognition](https://mlsanity.com/paper/2303.12766/) (2023)
- [Auxiliary Tasks in Multi-task Learning](https://mlsanity.com/paper/1805.06334/) (2018)
- [Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentation](https://mlsanity.com/paper/2103.14962/) (2021)
- [SuMa++: Efficient LiDAR-based Semantic SLAM](https://mlsanity.com/paper/2105.11320/) (2021)
- [Motion Guided Attention for Video Salient Object Detection](https://mlsanity.com/paper/1909.07061/) (2019)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception](https://mlsanity.com/paper/2109.05441/) (2021)
- [Multi-scale Interaction for Real-time LiDAR Data Segmentation on an Embedded Platform](https://mlsanity.com/paper/2008.09162/) (2020)
- [Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions](https://mlsanity.com/paper/2206.04129/) (2022)
- [Efficient Spatial-Temporal Information Fusion for LiDAR-Based 3D Moving Object Segmentation](https://mlsanity.com/paper/2207.02201/) (2022)
- [Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation](https://mlsanity.com/paper/2201.04501/) (2022)
- [LiDAR-based Recurrent 3D Semantic Segmentation with Temporal Memory Alignment](https://mlsanity.com/paper/2103.02263/) (2021)
- [Mask4Former: Mask Transformer for 4D Panoptic Segmentation](https://mlsanity.com/paper/2309.16133/) (2023)
- [MF-MOS: A Motion-Focused Model for Moving Object Segmentation](https://mlsanity.com/paper/2401.17023/) (2024)
- [MarS3D: A Plug-and-Play Motion-Aware Model for Semantic Segmentation on Multi-Scan 3D Point Clouds](https://mlsanity.com/paper/2307.09316/) (2023)
- [InsMOS: Instance-Aware Moving Object Segmentation in LiDAR Data](https://mlsanity.com/paper/2303.03909/) (2023)
- [LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentation](https://mlsanity.com/paper/2111.04875/) (2021)
- [SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2308.13323/) (2023)
- [MemorySeg: Online LiDAR Semantic Segmentation with a Latent Memory](https://mlsanity.com/paper/2311.01556/) (2023)
- [G3Reg: Pyramid Graph-based Global Registration using Gaussian Ellipsoid Model](https://mlsanity.com/paper/2308.11573/) (2023)

## Built on by (most cited first, 5 of 5)

- [Environment-Driven Online LiDAR-Camera Extrinsic Calibration](https://mlsanity.com/paper/2502.00801/) (2025)
- [Single-Frame Point-Pixel Registration via Supervised Cross-Modal Feature Matching](https://mlsanity.com/paper/2506.22784/) (2025)
- [DAGLFNet: Deep Feature Attention Guided Global and Local Feature Fusion for Pseudo-Image Point Cloud Segmentation](https://mlsanity.com/paper/2510.10471/) (2025)
- [RePL: Pseudo-label Refinement for Semi-supervised LiDAR Semantic Segmentation](https://mlsanity.com/paper/2604.06825/) (2026)
- [Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models](https://mlsanity.com/paper/2608.25418/) (2026)

## Abstract (arXiv)

4D LiDAR semantic segmentation, also referred to as multi-scan semantic segmentation, plays a crucial role in enhancing the environmental understanding capabilities of autonomous vehicles or robots. It classifies the semantic category of each LiDAR measurement point and detects whether it is dynamic, a critical ability for tasks like obstacle avoidance and autonomous navigation. Existing approaches often rely on computationally heavy 4D convolutions or recursive networks, which result in poor real-time performance, making them unsuitable for online robotics and autonomous driving applications. In this paper, we introduce SegNet4D, a novel real-time 4D semantic segmentation network offering both efficiency and strong semantic understanding. SegNet4D addresses 4D segmentation as two tasks: single-scan semantic segmentation and moving object segmentation, each tackled by a separate network head. Both results are combined in a motion-semantic fusion module to achieve comprehensive 4D segmentation. Additionally, instance information is extracted from the current scan and exploited for instance-wise segmentation consistency. Our approach surpasses state-of-the-art in both multi-scan semantic segmentation and moving object segmentation while offering greater efficiency, enabling real-time operation. Besides, its effectiveness and efficiency have also been validated on a real-world unmanned ground platform. Our code will be released at https://github.com/nubot-nudt/SegNet4D.