# Online Segmentation of LiDAR Sequences: Dataset and Algorithm

arXiv: https://arxiv.org/abs/2206.08194 · HTML: https://mlsanity.com/paper/2206.08194/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 57 references that are in the map.
- First posted on arXiv 16 Jun 2022.

## Builds on (most cited first, 30 of 41)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation](https://mlsanity.com/paper/2011.10033/) (2020)
- [CMT: Convolutional Neural Networks Meet Vision Transformers](https://mlsanity.com/paper/2107.06263/) (2021)
- [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [Rethinking and Improving Relative Position Encoding for Vision Transformer](https://mlsanity.com/paper/2107.14222/) (2021)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network](https://mlsanity.com/paper/2102.04530/) (2021)

## Built on by (most cited first, 14 of 14)

- [Efficient 3D Semantic Segmentation with Superpoint Transformer](https://mlsanity.com/paper/2306.08045/) (2023)
- [Domain generalization of 3D semantic segmentation in autonomous driving](https://mlsanity.com/paper/2212.04245/) (2022)
- [Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering](https://mlsanity.com/paper/2401.06704/) (2024)
- [Navya3DSeg -- Navya 3D Semantic Segmentation Dataset & split generation for autonomous vehicles](https://mlsanity.com/paper/2302.08292/) (2023)
- [Lidar Panoptic Segmentation in an Open World](https://mlsanity.com/paper/2409.14273/) (2024)
- [ParisLuco3D: A high-quality target dataset for domain generalization of LiDAR perception](https://mlsanity.com/paper/2310.16542/) (2023)
- [ZAHA: Introducing the Level of Facade Generalization and the Large-Scale Point Cloud Facade Semantic Segmentation Benchmark Dataset](https://mlsanity.com/paper/2411.04865/) (2024)
- [FLARES: Fast and Accurate LiDAR Multi-Range Semantic Segmentation](https://mlsanity.com/paper/2502.09274/) (2025)
- [TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding](https://mlsanity.com/paper/2511.07007/) (2025)
- [EZ-SP: Fast and Lightweight Superpoint-Based 3D Segmentation](https://mlsanity.com/paper/2512.00385/) (2025)
- [Semantic Segmentation on 3D Point Clouds with High Density Variations](https://mlsanity.com/paper/2307.01489/) (2023)
- [UnderOneFacade: Worldwide Facade Semantic Segmentation Benchmark Dataset](https://mlsanity.com/paper/2607.02018/) (2026)
- [Low Latency Instance Segmentation by Continuous Clustering for LiDAR Sensors](https://mlsanity.com/paper/2311.13976/) (2023)
- [ARCH2S: Dataset, Benchmark and Challenges for Learning Exterior Architectural Structures from Point Clouds](https://mlsanity.com/paper/2406.01337/) (2024)

## Abstract (arXiv)

Roof-mounted spinning LiDAR sensors are widely used by autonomous vehicles. However, most semantic datasets and algorithms used for LiDAR sequence segmentation operate on 360^circ frames, causing an acquisition latency incompatible with real-time applications. To address this issue, we first introduce HelixNet, a 10 billion point dataset with fine-grained labels, timestamps, and sensor rotation information necessary to accurately assess the real-time readiness of segmentation algorithms. Second, we propose Helix4D, a compact and efficient spatio-temporal transformer architecture specifically designed for rotating LiDAR sequences. Helix4D operates on acquisition slices corresponding to a fraction of a full sensor rotation, significantly reducing the total latency. Helix4D reaches accuracy on par with the best segmentation algorithms on HelixNet and SemanticKITTI with a reduction of over 5× in terms of latency and 50× in model size. The code and data are available at: https://romainloiseau.fr/helixnet