mlsanityData to 7 Oct 2026

PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation

arXiv 2312.04016 ↗First posted 7 Dec 2023cs.CVcited by 27

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

This paper proposes a cross-modal distillation framework, PartDistill, which transfers 2D knowledge from vision-language models (VLMs) to facilitate 3D shape part segmentation. PartDistill addresses three major challenges in this task: the lack of 3D segmentation in invisible or undetected regions in the 2D projections, inconsistent 2D predictions by VLMs, and the lack of knowledge accumulation across different 3D shapes. PartDistill consists of a teacher network that uses a VLM to make 2D predictions and a student network that learns from the 2D predictions while extracting geometrical features from multiple 3D shapes to carry out 3D part segmentation. A bi-directional distillation, including forward and backward distillations, is carried out within the framework, where the former forward distills the 2D predictions to the student network, and the latter improves the quality of the 2D predictions, which subsequently enhances the final 3D segmentation. Moreover, PartDistill can exploit generative models that facilitate effortless 3D shape creation for generating knowledge sources to be distilled. Through extensive experiments, PartDistill boosts the existing methods with substantial margins on widely used ShapeNetPart and PartNetE datasets, by more than 15% and 12% higher mIoU scores, respectively. The code for this work is available at https://github.com/ardianumam/PartDistill.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 42 of its 59 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

42 of 59 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021PointCLIP: Point Cloud Understanding by CLIP4 Dec

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  9. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  10. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  11. 2021Grounded Language-Image Pre-trainingcited by 1,147
  12. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
29 more
  1. 2022Language-driven Semantic Segmentationcited by 711
  2. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  3. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  4. 2021Image Segmentation Using Text and Image Promptscited by 509
  5. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432
  6. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 384
  7. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  8. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  9. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  10. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  11. 2021Shape As Points: A Differentiable Poisson Solvercited by 186
  12. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  13. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  14. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123
  15. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
  16. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generationcited by 110
  17. 2020Few-shot 3D Point Cloud Semantic Segmentationcited by 77
  18. 20163D Shape Segmentation with Projective Convolutional Networkscited by 63
  19. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  20. 2021Learning 3D Semantic Segmentation with only 2D Image Supervisioncited by 60
  21. 2019Cross-modal knowledge distillation for action recognitioncited by 44
  22. 2021Zero-Shot Learning on 3D Point Cloud Objects and Beyondcited by 38
  23. 2023Self-Supervised Image-to-Point Distillation via Semantically Tolerant Contrastive Losscited by 35
  24. 2020Label-Efficient Learning on Point Clouds using Approximate Convex Decompositionscited by 27
  25. 2021Double Similarity Distillation for Semantic Image Segmentationcited by 19
  26. 2022NeuForm: Adaptive Overfitting for Neural Shape Editingcited by 18
  27. 2021A Revisit of Shape Editing Techniques: from the Geometric to the Neural Viewpointcited by 16
  28. 20232D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervisioncited by 16
  29. 2022Data Efficient 3D Learner via Knowledge Transferred from 2D Modelcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 15; 2026: 9.

27 papers.

Sort:

By area:

  1. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 95
  2. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 91
  3. 2025HoloPart: Generative 3D Part Amodal Segmentationcited by 57
  4. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 42
  5. 2025Articulate AnyMesh: Open-Vocabulary 3D Articulated Objects Modelingcited by 41
  6. 2025P3-SAM: Native 3D Part Segmentationcited by 39
  7. 2024Find Any Part in 3Dcited by 30
  8. 20243D Part Segmentation via Geometric Aggregation of 2D Visual Featurescited by 15
  9. 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 15
  10. 2025GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentationcited by 10
  11. 2026SegviGen: Repurposing 3D Generative Model for Part Segmentationcited by 9
  12. 2025SPARK: Sim-ready Part-level Articulated Reconstruction with VLM Knowledgecited by 6
15 more
  1. 2025Open-Vocabulary Semantic Part Segmentation of 3D Humancited by 5
  2. 2026PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understandingcited by 5
  3. 2026MonoArt: Progressive Structural Reasoning for Monocular Articulated 3D Reconstructioncited by 5
  4. 2025Name That Part: 3D Part Segmentation and Namingcited by 3
  5. 2025Segment Any 3D-Part in a Scene from a Sentencecited by 2
  6. 2025Parameter-Efficient CLIP Adaptation for 3D Understanding via Unified Tokenizationcited by 1
  7. 2026PAR3D: A Unified 3D-MLLM with Part-Aware Representation for Scene Understandingcited by 1
  8. 2025MR-COSMO: Visual-Text Memory Recall and Direct CrOSs-MOdal Alignment Method for Query-Driven 3D Segmentationcited by 0
  9. 2025Adaptive Point-Prompt Tuning: Fine-Tuning Heterogeneous Foundation Models for 3D Point Cloud Analysiscited by 0
  10. 2025SegGraph: Leveraging Graphs of SAM Segments for Few-Shot 3D Part Segmentationcited by 0
  11. 2026CoSMo3D: Open-World Promptable 3D Semantic Part Segmentation through LLM-Guided Canonical Spatial Modelingcited by 0
  12. 2026EI-Part: Explode for Completion and Implode for Refinementcited by 0
  13. 2026Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshescited by 0
  14. 2026Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysiscited by 0
  15. 2026PartMat: Material-Aware 3D Part Decomposition with a Single Global Latentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Model
  2. 2025FOLK: Fast Open-Vocabulary 3D Instance Segmentation via Label-guided Knowledge Distillation
  3. 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Models
  4. 2024Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models
  5. 20253D-Aware Vision-Language Models Fine-Tuning with Geometric Distillation
  6. 2025PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding
  7. 2025CleverDistiller: Simple and Spatially Consistent Cross-modal Distillation
  8. 2025UniPart: Part-Level 3D Generation with Unified 3D Geom-Seg Latents