mlsanityData to 7 Oct 2026

Multi-Space Alignments Towards Universal LiDAR Segmentation

arXiv 2405.01538 ↗First posted 2 May 2024cs.CV cs.LG cs.ROcited by 24

Not on the map. Why?

Abstract (arXiv)

A unified and versatile LiDAR segmentation model with strong robustness and generalizability is desirable for safe autonomous driving perception. This work presents M3Net, a one-of-a-kind framework for fulfilling multi-task, multi-dataset, multi-modality LiDAR segmentation in a universal manner using just a single set of parameters. To better exploit data volume and diversity, we first combine large-scale driving datasets acquired by different types of sensors from diverse scenes and then conduct alignments in three spaces, namely data, feature, and label spaces, during the training. As a result, M3Net is capable of taming heterogeneous data for training state-of-the-art LiDAR segmentation models. Extensive experiments on twelve LiDAR segmentation datasets verify our effectiveness. Notably, using a shared set of parameters, M3Net achieves 75.1%, 83.1%, and 72.4% mIoU scores, respectively, on the official benchmarks of SemanticKITTI, nuScenes, and Waymo Open.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 109 of its 131 references that are in the map. First posted on arXiv 2 May 2024.

Builds on

109 of 131 references.

References · 109

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2017Mask R-CNNcited by 7,259
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  9. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  10. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  11. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  12. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
38 more
  1. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  2. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  3. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  4. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
  5. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  6. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  7. 2023Segment Everything Everywhere All at Oncecited by 546
  8. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  9. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  10. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  11. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 384
  12. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  13. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  14. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  15. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  16. 2021K-Net: Towards Unified Image Segmentationcited by 211
  17. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  18. 2023SegGPT: Segmenting Everything In Contextcited by 193
  19. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  20. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  21. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  22. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  23. 2021ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detectioncited by 158
  24. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 157
  25. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  26. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  27. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  28. 2021Simple multi-dataset detectioncited by 126
  29. 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
  30. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  31. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 126
  32. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 124
  33. 2020SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instancescited by 123
  34. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123
  35. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  36. 2020RELLIS-3D Dataset: Data, Benchmarks and Analysiscited by 118
  37. 2022Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentationcited by 118
  38. 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107

50 most cited of 109 All 109 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 11; 2026: 5.

24 papers.

Sort:

By area:

  1. 2024Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Drivingcited by 53
  2. 2024Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Drivingcited by 39
  3. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 32
  4. 2024Flow4D: Leveraging 4D Voxel Network for LiDAR Scene Flow Estimationcited by 22
  5. 2025LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenescited by 19
  6. 2025Perspective-Invariant 3D Object Detectioncited by 16
  7. 2024A Unified Framework for 3D Scene Understandingcited by 15
  8. 2024An Empirical Study of Training State-of-the-Art LiDAR Segmentation Modelscited by 12
  9. 2025SafeMap: Robust HD Map Construction from Incomplete Observationscited by 12
  10. 2025LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Drivingcited by 11
  11. 2025SPIRAL: Semantic-Aware Progressive LiDAR Scene Generation and Understandingcited by 10
  12. 2025La La LiDAR: Large-Scale Layout Generation from LiDAR Datacited by 10
12 more
  1. 2024One for All: Multi-Domain Joint Training for Point Cloud Based 3D Object Detectioncited by 8
  2. 20253EED: Ground Everything Everywhere in 3Dcited by 8
  3. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  4. 2024PanoSLAM: Panoptic 3D Scene Reconstruction via Gaussian SLAMcited by 3
  5. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3
  6. 2025Zero-Shot 3D Visual Grounding from Vision-Language Modelscited by 2
  7. 2025CitySeg: A 3D Open Vocabulary Semantic Segmentation Foundation Model in City-scale Scenarioscited by 2
  8. 2026Feasibility of Indoor Frame-Wise Lidar Semantic Segmentation via Distillation from Visual Foundation Modelcited by 1
  9. 2026RelFlexformer: Efficient Attention 3D-Transformers for Integrable Relative Positional Encodingscited by 1
  10. 2025UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehiclescited by 0
  11. 2026OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generationcited by 0
  12. 2026ZIL: Zero-shot Image-to-LiDAR Registrationcited by 0