mlsanityData to 14 Oct 2026

Novel class discovery meets foundation models for 3D semantic segmentation

arXiv 2312.03782 ↗First posted 6 Dec 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

The task of Novel Class Discovery (NCD) in semantic segmentation entails training a model able to accurately segment unlabelled (novel) classes, relying on the available supervision from annotated (base) classes. Although extensively investigated in 2D image data, the extension of the NCD task to the domain of 3D point clouds represents a pioneering effort, characterized by assumptions and challenges that are not present in the 2D case. This paper represents an advancement in the analysis of point cloud data in four directions. Firstly, it introduces the novel task of NCD for point cloud semantic segmentation. Secondly, it demonstrates that directly transposing the only existing NCD method for 2D image semantic segmentation to 3D data yields suboptimal results. Thirdly, a new NCD approach based on online clustering, uncertainty estimation, and semantic distillation is presented. Lastly, a novel evaluation protocol is proposed to rigorously assess the performance of NCD in point cloud semantic segmentation. Through comprehensive evaluations on the SemanticKITTI, SemanticPOSS, and S3DIS datasets, the paper demonstrates substantial superiority of the proposed method over the considered baselines.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 67 of its 86 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

67 of 86 references.

References · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2023Segment Anythingcited by 8,234
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  8. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  10. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  12. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
38 more
  1. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  2. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  3. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  4. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  5. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 585
  6. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  7. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  8. 2019Self-labelling via simultaneous clustering and representation learningcited by 528
  9. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  10. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 488
  11. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  12. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 455
  13. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  14. 2018PCN: Point Completion Networkcited by 384
  15. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 339
  16. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  17. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  18. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
  19. 2017SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloudcited by 232
  20. 2018SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloudcited by 228
  21. 2019Learning to Discover Novel Visual Categories via Deep Transfer Clusteringcited by 225
  22. 2017Submanifold Sparse Convolutional Networkscited by 223
  23. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  24. 2022Generalized Category Discoverycited by 211
  25. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  26. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  27. 20193D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutionscited by 173
  28. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 166
  29. 2019PU-GAN: a Point Cloud Upsampling Adversarial Networkcited by 164
  30. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  31. 2021A Unified Objective for Novel Class Discoverycited by 162
  32. 2018PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptorscited by 141
  33. 2020SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instancescited by 123
  34. 2021Neighborhood Contrastive Learning for Novel Class Discoverycited by 121
  35. 2020Point Cloud Completion by Skip-attention Network with Hierarchical Foldingcited by 106
  36. 2019PU-GCN: Point Cloud Upsampling using Graph Convolutional Networkscited by 101
  37. 2020OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in An Open Worldcited by 93
  38. 2020Self-supervised Learning of Point Clouds via Orientation Estimationcited by 72

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Novel Class Discovery for Point Cloud Segmentation via Joint Learning of Causal Representation and Reasoningcited by 2
  2. 2026WildIng: A Wildlife Image Invariant Representation Model for Geographical Domain Shiftcited by 1
  3. 2026Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentationcited by 1
  4. 2026Geometric-Aware Hypergraph Reasoning for Novel Class Discovery in Point Cloud Segmentationcited by 0