mlsanityData to 14 Oct 2026

LeAP: Consistent multi-domain 3D labeling using Foundation Models

arXiv 2502.03901 ↗First posted 6 Feb 2025cs.CV cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

Availability of datasets is a strong driver for research on 3D semantic understanding, and whilst obtaining unlabeled 3D point cloud data is straightforward, manually annotating this data with semantic labels is time-consuming and costly. Recently, Vision Foundation Models (VFMs) enable open-set semantic segmentation on camera images, potentially aiding automatic labeling. However,VFMs for 3D data have been limited to adaptations of 2D models, which can introduce inconsistencies to 3D labels. This work introduces Label Any Pointcloud (LeAP), leveraging 2D VFMs to automatically label 3D data with any set of classes in any kind of application whilst ensuring label consistency. Using a Bayesian update, point labels are combined into voxels to improve spatio-temporal consistency. A novel 3D Consistency Network (3D-CN) exploits 3D information to further improve label quality. Through various experiments, we show that our method can generate high-quality 3D semantic labels across diverse fields without any manual labeling. Further, models adapted to new domains using our labels show up to a 34.2 mIoU increase in semantic segmentation tasks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 61 of its 77 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

61 of 77 references.

References · 61

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2023Segment Anythingcited by 8,234
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  10. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  11. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  12. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
38 more
  1. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  2. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 575
  3. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  4. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  5. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  6. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
  7. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  8. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  9. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 409
  10. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  11. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  12. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
  13. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  14. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  15. 2018SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloudcited by 228
  16. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  17. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  18. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 181
  19. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  20. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  21. 2016SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networkscited by 135
  22. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 128
  23. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  24. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 125
  25. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 125
  26. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 104
  27. 2020S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Cloudscited by 102
  28. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 86
  29. 2022LidarMultiNet: Towards a Unified Multi-Task Network for LiDAR Perceptioncited by 74
  30. 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
  31. 2023PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigmcited by 56
  32. 2023Using a Waffle Iron for Automotive Point Cloud Semantic Segmentationcited by 53
  33. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 52
  34. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  35. 2022Learning to Detect Mobile Objects from LiDAR Scans Without Labelscited by 44
  36. 2023SAM3D: Zero-Shot 3D Object Detection via Segment Anything Modelcited by 40
  37. 2023Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Drivingcited by 37
  38. 2023Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Losscited by 35

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  2. 2026OccAny: Generalized Unconstrained Urban 3D Occupancycited by 6
  3. 2025LOSC: LiDAR Open-voc Segmentation Consolidatorcited by 5
  4. 20264DRC-OCC: Robust Semantic Occupancy Prediction Through Fusion of 4D Radar and Cameracited by 2
  5. 2025Label-Efficient LiDAR Panoptic Segmentationcited by 1
  6. 2026UniLiPs: Unified LiDAR Pseudo-Labeling with Geometry-Grounded Dynamic Scene Decompositioncited by 1
  7. 2026IGLOSS: Image Generation for Lidar Open-vocabulary Semantic Segmentationcited by 0
  8. 2026SplatLabel: Pseudo-Labelling through 4D Gaussian Splattingcited by 0