mlsanityData to 7 Oct 2026

2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds

arXiv 2207.04397 ↗First posted 10 Jul 2022cs.CVcited by 157

Not on the map. Why?

Abstract (arXiv)

As camera and LiDAR sensors capture complementary information used in autonomous driving, great efforts have been made to develop semantic segmentation algorithms through multi-modality data fusion. However, fusion-based approaches require paired data, i.e., LiDAR point clouds and camera images with strict point-to-pixel mappings, as the inputs in both training and inference, which seriously hinders their application in practical scenarios. Thus, in this work, we propose the 2D Priors Assisted Semantic Segmentation (2DPASS), a general training scheme, to boost the representation learning on point clouds, by fully taking advantage of 2D images with rich appearance. In practice, by leveraging an auxiliary modal fusion and multi-scale fusion-to-single knowledge distillation (MSFSKD), 2DPASS acquires richer semantic and structural information from the multi-modal data, which are then online distilled to the pure 3D network. As a result, equipped with 2DPASS, our baseline shows significant improvement with only point cloud inputs. Specifically, it achieves the state-of-the-arts on two large-scale benchmarks (i.e. SemanticKITTI and NuScenes), including top-1 results in both single and multiple scan(s) competitions of SemanticKITTI.

Key facts. Cited by 157 papers in the map (data through 7 Oct 2026). Builds on 62 of its 71 references that are in the map. First posted on arXiv 10 Jul 2022.

Builds on

62 of 71 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  9. 2016Pyramid Scene Parsing Networkcited by 2,472
  10. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  11. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  12. 2020Point Transformercited by 1,462
38 more
  1. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  2. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  3. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  4. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  5. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  6. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  7. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  8. 2013Do Deep Nets Really Need to be Deep?cited by 466
  9. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  10. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  11. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  12. 2019Relation-Shape Convolutional Neural Network for Point Cloud Analysiscited by 346
  13. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  14. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  15. 2017SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloudcited by 231
  16. 2018SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloudcited by 228
  17. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  18. 2020Sparse Single Sweep LiDAR Point Cloud Segmentation via Learning Contextual Shape Priors from Scene Completioncited by 222
  19. 2015Cross Modal Distillation for Supervision Transfercited by 173
  20. 2018Tangent Convolutions for Dense Prediction in 3Dcited by 173
  21. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  22. 2020SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentationcited by 170
  23. 2017Understanding Convolution for Semantic Segmentationcited by 168
  24. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  25. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  26. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  27. 2017Pointwise Convolutional Neural Networkscited by 126
  28. 2021SAT: 2D Semantics Assisted Training for 3D Visual Groundingcited by 124
  29. 2020Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentationcited by 112
  30. 2020AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentationcited by 87
  31. 2017Deep Projective 3D Semantic Segmentationcited by 81
  32. 2018Knowledge Transfer with Jacobian Matchingcited by 73
  33. 2015Efficient piecewise training of deep structured models for semantic segmentationcited by 71
  34. 2022X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioningcited by 70
  35. 2021Box-Aware Feature Enhancement for Single Object Tracking on Point Cloudscited by 66
  36. 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
  37. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  38. 20203D-MiniNet: Learning a 2D Representation from Point Clouds for Fast and Efficient 3D LIDAR Semantic Segmentationcited by 58

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 57; 2024: 43; 2025: 31; 2026: 19.

157 papers.

Sort:

By area:

  1. 2023Point Transformer V3: Simpler, Faster, Strongercited by 579
  2. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  3. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 143
  4. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  5. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 126
  6. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 124
  7. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  8. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 115
  9. 2024SparseOcc: Rethinking Sparse Latent Representation for Vision-Based Semantic Occupancy Predictioncited by 94
  10. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 85
  11. 2023SCPNet: Semantic Scene Completion on Point Cloudcited by 80
  12. 2024Point Cloud Mamba: Point Cloud Learning via State Space Modelcited by 66
38 more
  1. 2023UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebasecited by 57
  2. 2023Towards Label-free Scene Understanding by Vision Foundation Modelscited by 54
  3. 2023Using a Waffle Iron for Automotive Point Cloud Semantic Segmentationcited by 53
  4. 2024POP-3D: Open-Vocabulary 3D Occupancy Prediction from Imagescited by 52
  5. 2023FRNet: Frustum-Range Networks for Scalable LiDAR Segmentationcited by 50
  6. 2023Three Pillars improving Vision Foundation Model Distillation for Lidarcited by 43
  7. 2023GrowSP: Unsupervised Semantic Segmentation of 3D Point Cloudscited by 34
  8. 2023Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentationcited by 33
  9. 2025DINO in the Room: Leveraging 2D Foundation Models for 3D Segmentationcited by 31
  10. 2024Learning Modality-agnostic Representation for Semantic Segmentation from Any Modalitiescited by 29
  11. 2023Towards Robust Robot 3D Perception in Urban Environments: The UT Campus Object Datasetcited by 28
  12. 2023LinK: Linear Kernel for LiDAR-based 3D Perceptioncited by 26
  13. 2024MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillationcited by 26
  14. 2022Domain generalization of 3D semantic segmentation in autonomous drivingcited by 25
  15. 2022Let Images Give You More:Point Cloud Cross-Modal Training for Shape Analysiscited by 24
  16. 2023HypLiLoc: Towards Effective LiDAR Pose Regression with Hyperbolic Fusioncited by 24
  17. 2023Advancements in Point Cloud Data Augmentation for Deep Learning: A Surveycited by 24
  18. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  19. 2024Multi-Space Alignments Towards Universal LiDAR Segmentationcited by 24
  20. 2023See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Datacited by 21
  21. 2024Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentationcited by 21
  22. 2023A Survey of Label-Efficient Deep Learning for 3D Point Cloudscited by 20
  23. 2023Learning to Adapt SAM for Segmenting Cross-domain Point Cloudscited by 20
  24. 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
  25. 2023MarS3D: A Plug-and-Play Motion-Aware Model for Semantic Segmentation on Multi-Scan 3D Point Cloudscited by 19
  26. 2023Benchmarking the Robustness of LiDAR Semantic Segmentation Modelscited by 18
  27. 2023LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillationcited by 18
  28. 2024LiCROcc: Teach Radar for Accurate Semantic Occupancy Prediction using LiDAR and Cameracited by 18
  29. 2023MoPA: Multi-Modal Prior Aided Domain Adaptation for 3D Semantic Segmentationcited by 17
  30. 20232D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervisioncited by 16
  31. 2023ConDaFormer: Disassembled Transformer with Local Structure Enhancement for 3D Point Cloud Understandingcited by 16
  32. 2024LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernelscited by 15
  33. 2024RAPiD-Seg: Range-Aware Pointwise Distance Distribution Networks for 3D LiDAR Segmentationcited by 15
  34. 2023Exploiting the Complementarity of 2D and 3D Networks to Address Domain-Shift in 3D Semantic Segmentationcited by 14
  35. 2023LiDAR-UDA: Self-ensembling Through Time for Unsupervised LiDAR Domain Adaptationcited by 14
  36. 2023TFNet: Exploiting Temporal Cues for Fast and Accurate LiDAR Semantic Segmentationcited by 13
  37. 2023MemorySeg: Online LiDAR Semantic Segmentation with a Latent Memorycited by 13
  38. 2024SFPNet: Sparse Focal Point Network for Semantic Segmentation on General LiDAR Point Cloudscited by 13

50 most cited of 157 All 157 (JSON)