Training-Free Open-Vocabulary 3D Point-Cloud Segmentation on the Generalized Few-Shot Benchmark
Not on the map. Why?
Abstract (arXiv)
Generalized few-shot 3D point-cloud segmentation (GFS-PCS) asks a model to segment a scene into many base classes seen at training time and a set of novel classes. The state of the art reaches novel classes by reconciling a dense but noisy 3D vision-language prior with the few-shot support, but it pays for this with base 3D labels, per-episode training, and the support annotations themselves. We ask how far the same reconciliation can go with none of these: no training, no 3D labels, and not even the few-shot support. We pair a frozen 3D vision-language model (RegionPLC) as a dense prior with a frozen promptable concept segmenter (SAM3), prompted by the bare novel class names and lifted from posed RGB views, and reconcile the two by cross-view consistency: a point becomes novel only when enough of the views that see it agree. On the ScanNet200 GFS-PCS benchmark this fully training-free, open-vocabulary pipeline improves novel mIoU by +2.6 over the training-free dense prior while holding base accuracy within 0.5, and recovers a third (33%) of the novel-class gap to the trained state of the art that uses far more supervision. We further show that injecting the few-shot support into the pipeline, as a fusion gate and as a prototypical dense classifier, adds nothing over consistency alone and in fact degrades it through the classifier, which is why the method needs no support at all. On the harder ScanNet++ benchmark, where the dense prior is far weaker on novel classes, the same pipeline nearly doubles novel mIoU (+15.7, from 16.2 to 31.9) at a 1.7 base cost, lifting the harmonic mean from 21.5 to 31.1
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 16 Jul 2026.
Builds on
44 of 51 references.
References · 44
- 2023Segment Anythingcited by 8,195
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,498
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,096
- 2017Prototypical Networks for Few-shot Learningcited by 3,011
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,820
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,205
- 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,203
- 2021Grounded Language-Image Pre-trainingcited by 1,152
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,082
- 2025SAM 3: Segment Anything with Conceptscited by 865
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 752
- 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 747
32 more
- 2022Language-driven Semantic Segmentationcited by 715
- 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
- 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 613
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 584
- 2021Extract Free Dense Labels from CLIPcited by 582
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 557
- 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
- 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
- 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 244
- 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
- 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 176
- 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 166
- 2022CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-trainingcited by 159
- 2021Mix3D: Out-of-Context Data Augmentation for 3D Scenescited by 126
- 2020PointMixup: Augmentation for Point Cloudscited by 104
- 2023RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understandingcited by 93
- 2023CLIP-FO3D: Learning Free Open-world 3D Scene Representations from 2D Dense CLIPcited by 89
- 2023OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentationcited by 82
- 2020Few-shot 3D Point Cloud Semantic Segmentationcited by 77
- 2020Generalized Few-shot Semantic Segmentationcited by 59
- 2023Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understandingcited by 38
- 2025Mosaic3D: Foundation Dataset and Model for Open-Vocabulary 3D Segmentationcited by 21
- 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
- 2023Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancementcited by 19
- 2024Rethinking Few-shot 3D Point Cloud Semantic Segmentationcited by 18
- 2024No Time to Train: Empowering Non-Parametric Networks for Few-shot 3D Scene Segmentationcited by 18
- 2025Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Modelcited by 17
- 2024UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representationcited by 15
- 2024Dense Multimodal Alignment for Open-Vocabulary 3D Scene Understandingcited by 15
- 2024Segment Any 3D Object with Languagecited by 13
- 2023Generalized Few-Shot Point Cloud Segmentation Via Geometric Wordscited by 12
- 2024Open-Vocabulary SAM3D: Towards Training-free Open-Vocabulary 3D Scene Understandingcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025PGOV3D: Open-Vocabulary 3D Semantic Segmentation with Partial-to-Global Curriculum
- 20253D-PointZshotS: Geometry-Aware 3D Point Cloud Zero-Shot Semantic Segmentation Narrowing the Visual-Semantic Gap
- 2025All in One: Visual-Description-Guided Unified Point Cloud Segmentation
- 2023Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment
- 2023Weakly Supervised 3D Open-vocabulary Segmentation
- 2025Just Add Geometry: Gradient-Free Open-Vocabulary 3D Detection Without Human-in-the-Loop
- 2025From Dataset to Real-world: General 3D Object Detection via Generalized Cross-domain Few-shot Learning
- 2025Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation