mlsanityData to 7 Oct 2026

3x2: 3D Object Part Segmentation by 2D Semantic Correspondences

arXiv 2407.09648 ↗First posted 12 Jul 2024cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

3D object part segmentation is essential in computer vision applications. While substantial progress has been made in 2D object part segmentation, the 3D counterpart has received less attention, in part due to the scarcity of annotated 3D datasets, which are expensive to collect. In this work, we propose to leverage a few annotated 3D shapes or richly annotated 2D datasets to perform 3D object part segmentation. We present our novel approach, termed 3-By-2 that achieves SOTA performance on different benchmarks with various granularity levels. By using features from pretrained foundation models and exploiting semantic and geometric correspondences, we are able to overcome the challenges of limited 3D annotations. Our approach leverages available 2D labels, enabling effective 3D object part segmentation. Our method 3-By-2 can accommodate various part taxonomies and granularities, demonstrating interesting part label transfer ability across different object categories. Project website: urlhttps://ngailapdi.github.io/projects/3by2/.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 45 of its 56 references that are in the map. First posted on arXiv 12 Jul 2024.

Builds on

45 of 56 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Segment Anythingcited by 8,158
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  7. 2021Grounded Language-Image Pre-trainingcited by 1,147
  8. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  9. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  10. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  11. 2023Emergent Correspondence from Image Diffusioncited by 437
  12. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432
33 more
  1. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  2. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  3. 2023OpenMask3D: Open-Vocabulary 3D Instance Segmentationcited by 288
  4. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 197
  5. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 186
  6. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  7. 2023PACO: Parts and Attributes of Common Objectscited by 146
  8. 2019SPair-71k: A Large-scale Benchmark for Semantic Correspondencecited by 140
  9. 2023Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidancecited by 123
  10. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  11. 20183DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentationcited by 118
  12. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
  13. 2021PartImageNet: A Large, High-Quality Dataset of Partscited by 107
  14. 20213D AffordanceNet: A Benchmark for Visual Object Affordance Understandingcited by 100
  15. 2020Few-shot 3D Point Cloud Semantic Segmentationcited by 77
  16. 2019Multi-view PointNet for 3D Scene Understandingcited by 76
  17. 2018Hierarchical interpretations for neural network predictionscited by 66
  18. 2023Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labelscited by 64
  19. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  20. 2023Going Denser with Open-Vocabulary Part Segmentationcited by 51
  21. 2023SAMPro3D: Locating SAM Prompts in 3D for Zero-Shot Instance Segmentationcited by 44
  22. 2020Unsupervised Learning of Intrinsic Structural Representation Pointscited by 39
  23. 2023PartSLIP++: Enhancing Low-Shot 3D Part Segmentation via Multi-View Instance Segmentation and Maximum Likelihood Estimationcited by 37
  24. 2020Label-Efficient Learning on Point Clouds using Approximate Convex Decompositionscited by 27
  25. 2023IKEA-Manual: Seeing Shape Assembly Step by Stepcited by 22
  26. 2023ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentationcited by 18
  27. 2024PartSTAD: 2D-to-3D Part Segmentation Task Adaptationcited by 18
  28. 2024Composable Part-Based Manipulationcited by 17
  29. 2022AutoGPart: Intermediate Supervision Search for Generalizable 3D Part Segmentationcited by 10
  30. 2022MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentationcited by 10
  31. 2023When 3D Bounding-Box Meets SAM: Point Cloud Instance Segmentation with Weak-and-Noisy Supervisioncited by 9
  32. 2021Similarity-Aware Fusion Network for 3D Semantic Segmentationcited by 3
  33. 2023The Sum of Its Parts: Visual Part Segmentation for Inertial Parameter Identification of Manipulated Objectscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 7; 2026: 2.

10 papers.

  1. 2024SAMPart3D: Segment Any Part in 3D Objectscited by 95
  2. 2025HoloPart: Generative 3D Part Amodal Segmentationcited by 57
  3. 2025X-Part: high fidelity and structure coherent shape decompositioncited by 42
  4. 2025MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Cloudscited by 15
  5. 2025Do It Yourself: Learning Semantic Correspondence from Pseudo-Labelscited by 13
  6. 2026HY3D-Bench: Generation of 3D Assetscited by 11
  7. 2025LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Cloudscited by 8
  8. 2026Artiverse: A Diverse and Physically Grounded Dataset for Articulated Objectscited by 4
  9. 2025Semantic Correspondence: Unified Benchmarking and a Strong Baselinecited by 3
  10. 2025SegGraph: Leveraging Graphs of SAM Segments for Few-Shot 3D Part Segmentationcited by 0