mlsanityData to 14 Oct 2026

SATR: Zero-Shot Semantic Segmentation of 3D Shapes

arXiv 2304.04909 ↗First posted 11 Apr 2023cs.CVcited by 62

Not on the map. Why?

Abstract (arXiv)

We explore the task of zero-shot semantic segmentation of 3D shapes by using large-scale off-the-shelf 2D image recognition models. Surprisingly, we find that modern zero-shot 2D object detectors are better suited for this task than contemporary text/image similarity predictors or even zero-shot 2D segmentation networks. Our key finding is that it is possible to extract accurate 3D segmentation maps from multi-view bounding box predictions by using the topological properties of the underlying surface. For this, we develop the Segmentation Assignment with Topological Reweighting (SATR) algorithm and evaluate it on ShapeNetPart and our proposed FAUST benchmarks. SATR achieves state-of-the-art performance and outperforms a baseline algorithm by 1.3% and 4% average mIoU on the FAUST coarse and fine-grained benchmarks, respectively, and by 5.2% average mIoU on the ShapeNetPart benchmark. Our source code and data will be publicly released. Project webpage: https://samir55.github.io/SATR/.

Key facts. Cited by 62 papers in the map (data through 14 Oct 2026). Builds on 77 of its 102 references that are in the map. First posted on arXiv 11 Apr 2023.

Builds on

77 of 102 references.

References · 77

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
38 more
  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  5. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  6. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  7. 2021Grounded Language-Image Pre-trainingcited by 1,156
  8. 2016Modeling Context in Referring Expressionscited by 1,063
  9. 2021Scaling Vision Transformerscited by 999
  10. 2022Language-driven Semantic Segmentationcited by 717
  11. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  12. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  13. 2021Image Segmentation Using Text and Image Promptscited by 512
  14. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 473
  15. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  16. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  17. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  18. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 334
  19. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  20. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 300
  21. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  22. 2022GLIPv2: Unifying Localization and Vision-Language Understandingcited by 275
  23. 2022Language-Grounded Indoor 3D Semantic Segmentation in the Wildcited by 246
  24. 2019Zero-Shot Semantic Segmentationcited by 245
  25. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  26. 2021Decoupling Zero-Shot Semantic Segmentationcited by 199
  27. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  28. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  29. 2018Zero-Shot Object Detectioncited by 189
  30. 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
  31. 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 177
  32. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 174
  33. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 165
  34. 2022SegCLIP: Patch Aggregation with Learnable Centers for Open-Vocabulary Semantic Segmentationcited by 164
  35. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  36. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 146
  37. 2022Learning to Generate Text-grounded Mask for Open-world Semantic Segmentation from Only Image-Text Pairscited by 130
  38. 2022GRiT: A Generative Region-to-text Transformer for Object Understandingcited by 121

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 17; 2025: 19; 2026: 15.

62 papers.

Sort:

By area:

  1. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
  2. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  3. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 97
  4. 2023CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentationcited by 96
  5. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 95
  6. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  7. 2025OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesioncited by 61
  8. 2025HoloPart: Generative 3D Part Amodal Segmentationcited by 60
  9. 2024Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databasescited by 45
  10. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 44
  11. 2025P3-SAM: Native 3D Part Segmentationcited by 43
  12. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 35
38 more
  1. 2023Zero-Shot 3D Shape Correspondencecited by 33
  2. 2024Find Any Part in 3Dcited by 32
  3. 2024Search3D: Hierarchical Open-Vocabulary 3D Segmentationcited by 31
  4. 2023PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillationcited by 28
  5. 2025Particulate: Feed-Forward 3D Object Articulationcited by 23
  6. 2024MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesiscited by 22
  7. 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 19
  8. 2023ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentationcited by 18
  9. 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptationcited by 18
  10. 2023Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Featurescited by 17
  11. 2025PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Datacited by 16
  12. 2023CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programscited by 15
  13. 20243D Part Segmentation via Geometric Aggregation of 2D Visual Featurescited by 15
  14. 2024iSeg: Interactive 3D Segmentation via Interactive Attentioncited by 14
  15. 2025Hierarchical Neural Semantic Representation for 3D Semantic Correspondencecited by 14
  16. 2023Neural Semantic Surface Mapscited by 13
  17. 2025GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentationcited by 13
  18. 2024Integrating Efficient Optimal Transport and Functional Maps For Unsupervised Shape Correspondence Learningcited by 12
  19. 2024SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimationcited by 12
  20. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 11
  21. 2026HY3D-Bench: Generation of 3D Assetscited by 11
  22. 20243x2: 3D Object Part Segmentation by 2D Semantic Correspondencescited by 10
  23. 2023DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentationcited by 7
  24. 2025Textured 3D Regenerative Morphing with 3D Diffusion Priorcited by 6
  25. 2025ASIA: Adaptive 3D Segmentation using Few Image Annotationscited by 6
  26. 2026PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understandingcited by 6
  27. 2024On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Surveycited by 5
  28. 2025Open-Vocabulary Semantic Part Segmentation of 3D Humancited by 5
  29. 2026Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objectscited by 4
  30. 2026Instruct-Particulate: Scaling Feed-Forward 3D Object Articulation with Kinematic Controlcited by 4
  31. 2024Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Surveycited by 3
  32. 2025GenAnalysis: Joint Shape Analysis by Learning Man-Made Shape Generators with Deformation Regularizationscited by 3
  33. 20253D PixBrush: Image-Guided Local Texture Synthesiscited by 3
  34. 2024MM-Mixing: Multi-Modal Mixing Alignment for 3D Understandingcited by 2
  35. 2024Test-time Contrastive Concepts for Open-world Semantic Segmentation with Vision-Language Modelscited by 2
  36. 2025PASG: A Closed-Loop Framework for Automated Geometric Primitive Extraction and Semantic Anchoring in Robotic Manipulationcited by 1
  37. 2026Universal 3D Shape Matching via Coarse-to-Fine Language Guidancecited by 1
  38. 2026GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondencecited by 1

50 most cited of 62 All 62 (JSON)