# SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation

arXiv: https://arxiv.org/abs/2308.13323 · HTML: https://mlsanity.com/paper/2308.13323/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 51 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completion](https://mlsanity.com/paper/2012.03762/) (2020)
- [Tangent Convolutions for Dense Prediction in 3D](https://mlsanity.com/paper/1807.02443/) (2018)
- [End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/1910.06528/) (2019)
- [Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking](https://mlsanity.com/paper/2109.03805/) (2021)
- [(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network](https://mlsanity.com/paper/2102.04530/) (2021)
- [FLOT: Scene Flow on Point Clouds Guided by Optimal Transport](https://mlsanity.com/paper/2007.11142/) (2020)
- [Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2206.02099/) (2022)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps](https://mlsanity.com/paper/2003.06754/) (2020)
- [TPCN: Temporal Point Cloud Networks for Motion Forecasting](https://mlsanity.com/paper/2103.03067/) (2021)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion](https://mlsanity.com/paper/2103.07074/) (2021)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data](https://mlsanity.com/paper/2105.08971/) (2021)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception](https://mlsanity.com/paper/2109.05441/) (2021)
- [Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving](https://mlsanity.com/paper/2011.13628/) (2020)
- [An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/2007.12392/) (2020)

## Built on by (most cited first, 14 of 14)

- [RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentation](https://mlsanity.com/paper/2407.10159/) (2024)
- [TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation](https://mlsanity.com/paper/2407.09751/) (2024)
- [MutualForce: Mutual-Aware Enhancement for 4D Radar-LiDAR 3D Object Detection](https://mlsanity.com/paper/2501.10266/) (2025)
- [Zero-Shot 4D Lidar Panoptic Segmentation](https://mlsanity.com/paper/2504.00848/) (2025)
- [NSM4D: Neural Scene Model Based Online 4D Point Cloud Sequence Understanding](https://mlsanity.com/paper/2310.08326/) (2023)
- [SegNet4D: Efficient Instance-Aware 4D Semantic Segmentation for LiDAR Point Cloud](https://mlsanity.com/paper/2406.16279/) (2024)
- [nuScenes Revisited: Progress and Challenges in Autonomous Driving](https://mlsanity.com/paper/2512.02448/) (2025)
- [Learning from Spatio-temporal Correlation for Semi-Supervised LiDAR Semantic Segmentation](https://mlsanity.com/paper/2410.06893/) (2024)
- [P3P: Pseudo-3D Pre-training for Scaling 3D Voxel-based Masked Autoencoders](https://mlsanity.com/paper/2408.10007/) (2024)
- [4D-CS: Exploiting Cluster Prior for 4D Spatio-Temporal LiDAR Semantic Segmentation](https://mlsanity.com/paper/2501.02937/) (2025)
- [Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentation](https://mlsanity.com/paper/2509.00379/) (2025)
- [4DSegStreamer: Streaming 4D Panoptic Segmentation via Dual Threads](https://mlsanity.com/paper/2510.17664/) (2025)
- [UniD-Shift: Towards Unified Semantic Segmentation via Interpretable Share-Private Multimodal Decomposition](https://mlsanity.com/paper/2605.07356/) (2026)
- [MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers](https://mlsanity.com/paper/2610.12624/) (2026)

## Abstract (arXiv)

LiDAR-based semantic perception tasks are critical yet challenging for autonomous driving. Due to the motion of objects and static/dynamic occlusion, temporal information plays an essential role in reinforcing perception by enhancing and completing single-frame knowledge. Previous approaches either directly stack historical frames to the current frame or build a 4D spatio-temporal neighborhood using KNN, which duplicates computation and hinders realtime performance. Based on our observation that stacking all the historical points would damage performance due to a large amount of redundant and misleading information, we propose the Sparse Voxel-Adjacent Query Network (SVQNet) for 4D LiDAR semantic segmentation. To take full advantage of the historical frames high-efficiently, we shunt the historical points into two groups with reference to the current points. One is the Voxel-Adjacent Neighborhood carrying local enhancing knowledge. The other is the Historical Context completing the global knowledge. Then we propose new modules to select and extract the instructive features from the two groups. Our SVQNet achieves state-of-the-art performance in LiDAR semantic segmentation of the SemanticKITTI benchmark and the nuScenes dataset.