Rethinking Multimodal Few-Shot 3D Point Cloud Segmentation: From Fused Refinement to Decoupled Arbitration
Not on the map. Why?
Abstract (arXiv)
In this paper, we revisit multimodal few-shot 3D point cloud semantic segmentation (FS-PCS), identifying a conflict in "Fuse-then-Refine" paradigms: the "Plasticity-Stability Dilemma." In addition, CLIP's inter-class confusion can result in semantic blindness. To address these issues, we present the Decoupled-experts Arbitration Few-Shot SegNet (DA-FSS), a model that effectively distinguishes between semantic and geometric paths and mutually regularizes their gradients to achieve better generalization. DA-FSS employs the same backbone and pre-trained text encoder as MM-FSS to generate text embeddings, which can increase free modalities' utilization rate and better leverage each modality's information space. To achieve this, we propose a Parallel Expert Refinement module to generate each modal correlation. We also propose a Stacked Arbitration Module (SAM) to perform convolutional fusion and arbitrate correlations for each modality pathway. The Parallel Experts decouple two paths: a Geometric Expert maintains plasticity, and a Semantic Expert ensures stability. They are coordinated via a Decoupled Alignment Module (DAM) that transfers knowledge without propagating confusion. Experiments on popular datasets (S3DIS, ScanNet) demonstrate the superiority of DA-FSS over MM-FSS. Meanwhile, geometric boundaries, completeness, and texture differentiation are all superior to the baseline. The code is available at: https://github.com/MoWenQAQ/DA-FSS/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 42 references that are in the map. First posted on arXiv 4 Jan 2026.
Builds on
33 of 42 references.
References · 33
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
- 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
- 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
- 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
- 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
- 2022Decoupled Knowledge Distillationcited by 388
- 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 362
- 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
21 more
- 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 267
- 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
- 2023ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understandingcited by 202
- 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 194
- 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 181
- 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 177
- 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 167
- 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 124
- 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
- 2023RegionPLC: Regional Point-Language Contrastive Learning for Open-World 3D Scene Understandingcited by 94
- 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
- 2020Few-shot 3D Point Cloud Semantic Segmentationcited by 77
- 2024A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)cited by 44
- 2023Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Modelscited by 42
- 2023A Survey of Label-Efficient Deep Learning for 3D Point Cloudscited by 20
- 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
- 2023Boosting Few-shot 3D Point Cloud Segmentation via Query-Guided Enhancementcited by 19
- 2024Rethinking Few-shot 3D Point Cloud Semantic Segmentationcited by 18
- 2025Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Modelcited by 17
- 2023Few-Shot 3D Point Cloud Semantic Segmentation via Stratified Class-Specific Attention Based Transformer Networkcited by 16
- 2024UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representationcited by 15
Built on by
Not cited yet.