# SATR: Zero-Shot Semantic Segmentation of 3D Shapes

arXiv: https://arxiv.org/abs/2304.04909 · HTML: https://mlsanity.com/paper/2304.04909/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- Builds on 77 of its 102 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 30 of 77)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [SLIP: Self-supervision meets Language-Image Pre-training](https://mlsanity.com/paper/2112.12750/) (2021)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)

## Built on by (most cited first, 30 of 62)

- [OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding](https://mlsanity.com/paper/2305.10764/) (2023)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentation](https://mlsanity.com/paper/2312.12359/) (2023)
- [PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond](https://mlsanity.com/paper/2504.11451/) (2025)
- [DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment](https://mlsanity.com/paper/2412.16334/) (2024)
- [OmniPart: Part-Aware 3D Generation with Semantic Decoupling and Structural Cohesion](https://mlsanity.com/paper/2507.06165/) (2025)
- [HoloPart: Generative 3D Part Amodal Segmentation](https://mlsanity.com/paper/2504.07943/) (2025)
- [Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases](https://mlsanity.com/paper/2403.09675/) (2024)
- [X-Part: high fidelity and structure coherent shape decomposition](https://mlsanity.com/paper/2509.08643/) (2025)
- [P3-SAM: Native 3D Part Segmentation](https://mlsanity.com/paper/2509.06784/) (2025)
- [3D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillation](https://mlsanity.com/paper/2311.09571/) (2023)
- [Zero-Shot 3D Shape Correspondence](https://mlsanity.com/paper/2306.03253/) (2023)
- [Find Any Part in 3D](https://mlsanity.com/paper/2411.13550/) (2024)
- [Search3D: Hierarchical Open-Vocabulary 3D Segmentation](https://mlsanity.com/paper/2409.18431/) (2024)
- [PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation](https://mlsanity.com/paper/2312.04016/) (2023)
- [Particulate: Feed-Forward 3D Object Articulation](https://mlsanity.com/paper/2512.11798/) (2025)
- [MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis](https://mlsanity.com/paper/2407.13675/) (2024)
- [MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds](https://mlsanity.com/paper/2508.14879/) (2025)
- [ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentation](https://mlsanity.com/paper/2311.14262/) (2023)
- [PartSTAD: 2D-to-3D Part Segmentation Task Adaptation](https://mlsanity.com/paper/2401.05906/) (2024)
- [Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features](https://mlsanity.com/paper/2311.18113/) (2023)
- [PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data](https://mlsanity.com/paper/2509.21965/) (2025)
- [CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs](https://mlsanity.com/paper/2311.16703/) (2023)
- [3D Part Segmentation via Geometric Aggregation of 2D Visual Features](https://mlsanity.com/paper/2412.04247/) (2024)
- [iSeg: Interactive 3D Segmentation via Interactive Attention](https://mlsanity.com/paper/2404.03219/) (2024)
- [Hierarchical Neural Semantic Representation for 3D Semantic Correspondence](https://mlsanity.com/paper/2509.17431/) (2025)
- [Neural Semantic Surface Maps](https://mlsanity.com/paper/2309.04836/) (2023)
- [GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation](https://mlsanity.com/paper/2508.14036/) (2025)
- [Integrating Efficient Optimal Transport and Functional Maps For Unsupervised Shape Correspondence Learning](https://mlsanity.com/paper/2403.01781/) (2024)

## Abstract (arXiv)

We explore the task of zero-shot semantic segmentation of 3D shapes by using large-scale off-the-shelf 2D image recognition models. Surprisingly, we find that modern zero-shot 2D object detectors are better suited for this task than contemporary text/image similarity predictors or even zero-shot 2D segmentation networks. Our key finding is that it is possible to extract accurate 3D segmentation maps from multi-view bounding box predictions by using the topological properties of the underlying surface. For this, we develop the Segmentation Assignment with Topological Reweighting (SATR) algorithm and evaluate it on ShapeNetPart and our proposed FAUST benchmarks. SATR achieves state-of-the-art performance and outperforms a baseline algorithm by 1.3% and 4% average mIoU on the FAUST coarse and fine-grained benchmarks, respectively, and by 5.2% average mIoU on the ShapeNetPart benchmark. Our source code and data will be publicly released. Project webpage: https://samir55.github.io/SATR/.