# TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation

arXiv: https://arxiv.org/abs/2407.09751 · HTML: https://mlsanity.com/paper/2407.09751/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 45 references that are in the map.
- First posted on arXiv 13 Jul 2024.

## Builds on (most cited first, 30 of 40)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud](https://mlsanity.com/paper/1809.08495/) (2018)
- [Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion](https://mlsanity.com/paper/2012.03762/) (2020)
- [RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentation](https://mlsanity.com/paper/2103.12978/) (2021)
- [SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentation](https://mlsanity.com/paper/2004.01803/) (2020)
- [(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network](https://mlsanity.com/paper/2102.04530/) (2021)
- [Rethinking Range View Representation for LiDAR Segmentation](https://mlsanity.com/paper/2303.05367/) (2023)
- [Spherical Transformer for LiDAR-based 3D Recognition](https://mlsanity.com/paper/2303.12766/) (2023)
- [Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2206.02099/) (2022)
- [DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation](https://mlsanity.com/paper/2203.14211/) (2022)
- [LaserMix for Semi-Supervised LiDAR Semantic Segmentation](https://mlsanity.com/paper/2207.00026/) (2022)
- [YOLOP: You Only Look Once for Panoptic Driving Perception](https://mlsanity.com/paper/2108.11250/) (2021)
- [PolarMix: A General Data Augmentation Technique for LiDAR Point Clouds](https://mlsanity.com/paper/2208.00223/) (2022)
- [LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perception](https://mlsanity.com/paper/2209.09385/) (2022)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [Learning 3D Semantic Segmentation with only 2D Image Supervision](https://mlsanity.com/paper/2110.11325/) (2021)
- [UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase](https://mlsanity.com/paper/2309.05573/) (2023)
- [RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/1906.00208/) (2019)
- [LatticeNet: Fast Point Cloud Segmentation Using Permutohedral Lattices](https://mlsanity.com/paper/1912.05905/) (2019)
- [4D-StOP: Panoptic Segmentation of 4D LiDAR using Spatio-temporal Object Proposal Generation and Aggregation](https://mlsanity.com/paper/2209.14858/) (2022)
- [FuseSeg: LiDAR Point Cloud Segmentation Fusing Multi-Modal Data](https://mlsanity.com/paper/1912.08487/) (2019)

## Built on by (most cited first, 13 of 13)

- [Zero-Shot 4D Lidar Panoptic Segmentation](https://mlsanity.com/paper/2504.00848/) (2025)
- [Segment and Select: Vision-Language Segmentation in 3D Scenarios](https://mlsanity.com/paper/2606.10594/) (2026)
- [HexPlane Representation for 3D Semantic Scene Understanding](https://mlsanity.com/paper/2503.05127/) (2025)
- [SPPSFormer: High-quality Superpoint-based Transformer for Roof Plane Instance Segmentation from Point Clouds](https://mlsanity.com/paper/2505.24475/) (2025)
- [4D-CS: Exploiting Cluster Prior for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2501.02937/) (2025)
- [An Experimental Study of SOTA LiDAR Segmentation Models](https://mlsanity.com/paper/2502.12860/) (2025)
- [4DSegStreamer: Streaming 4D Panoptic Segmentation via Dual Threads](https://mlsanity.com/paper/2510.17664/) (2025)
- [UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition](https://mlsanity.com/paper/2605.07356/) (2026)
- [LOGOS: LiDAR-Only Gaussian Elevation Splatting for Unified Tiny Obstacle Segmentation](https://mlsanity.com/paper/2606.21527/) (2026)
- [Heterogeneous and Adept Snapshot Distillation for 3D Semantic Segmentation](https://mlsanity.com/paper/2606.25278/) (2026)
- [PVCap: Towards Accurate 3D Dense Captioning via PseudoCap and VoxelCapNet](https://mlsanity.com/paper/2607.06097/) (2026)
- [Bootstrapping a 4D LiDAR Annotation Tool from Video Foundation Models](https://mlsanity.com/paper/2608.25418/) (2026)
- [Privacy-Preserving Semantic Segmentation from High-Resolution Depth and Ultra-Low-Resolution RGB](https://mlsanity.com/paper/2609.28360/) (2026)

## Abstract (arXiv)

Training deep models for LiDAR semantic segmentation is challenging due to the inherent sparsity of point clouds. Utilizing temporal data is a natural remedy against the sparsity problem as it makes the input signal denser. However, previous multi-frame fusion algorithms fall short in utilizing sufficient temporal information due to the memory constraint, and they also ignore the informative temporal images. To fully exploit rich information hidden in long-term temporal point clouds and images, we present the Temporal Aggregation Network, termed TASeg. Specifically, we propose a Temporal LiDAR Aggregation and Distillation (TLAD) algorithm, which leverages historical priors to assign different aggregation steps for different classes. It can largely reduce memory and time overhead while achieving higher accuracy. Besides, TLAD trains a teacher injected with gt priors to distill the model, further boosting the performance. To make full use of temporal images, we design a Temporal Image Aggregation and Fusion (TIAF) module, which can greatly expand the camera FOV and enhance the present features. Temporal LiDAR points in the camera FOV are used as mediums to transform temporal image features to the present coordinate for temporal multi-modal fusion. Moreover, we develop a Static-Moving Switch Augmentation (SMSA) algorithm, which utilizes sufficient temporal information to enable objects to switch their motion states freely, thus greatly increasing static and moving training samples. Our TASeg ranks 1st on three challenging tracks, i.e., SemanticKITTI single-scan track, multi-scan track and nuScenes LiDAR segmentation track, strongly demonstrating the superiority of our method. Codes are available at https://github.com/LittlePey/TASeg.