mlsanityData to 14 Oct 2026

Best Segmentation Buddies for Image-Shape Correspondence

arXiv 2605.18193 ↗First posted 18 May 2026cs.CV cs.GRcited by 0

Not on the map. Why?

Abstract (arXiv)

Finding correspondences is a fundamental and extensively researched problem in computer vision and graphics. In this work, we examine the underexplored task of estimating segmentation-to-segmentation correspondence between images in the wild and untextured 3D shapes. This task is highly challenging due to substantial differences in appearance, geometry, and viewpoint. Our approach bridges the cross-modality gap by linking pixels in the image segment to vertices in the corresponding semantic part of the 3D shape. To achieve this, we first distill deep visual features from a 2D vision model onto the 3D shape surface, allowing for the computation of feature similarity between image pixels and shape vertices. Then, we identify Best Segmentation Buddies, vertices whose most similar image pixel lies within the image segmentation region, enabling the reliable discovery of vertices in semantically corresponding shape parts. Finally, we leverage distilled 3D features from the 2D image segmentation model to segment the shape directly in 3D, bootstrapping the correspondence process. We demonstrate the generality and robustness of our approach across a wide range of image-shape pairs, showcasing accurate and semantically meaningful correspondences. Our project page is at https://threedle.github.io/bsb/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 55 of its 71 references that are in the map. First posted on arXiv 18 May 2026.

Builds on

55 of 71 references.

References · 55

  1. 2023Segment Anythingcited by 8,234
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  8. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  9. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  10. 2021Extract Free Dense Labels from CLIPcited by 583
  11. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  12. 2023Segment Everything Everywhere All at Oncecited by 548
38 more
  1. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  2. 2023Emergent Correspondence from Image Diffusioncited by 446
  3. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  4. 2016LIFT: Learned Invariant Feature Transformcited by 306
  5. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 301
  6. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  7. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 214
  8. 2017Working hard to know your neighbor's margins: Local descriptor learning losscited by 212
  9. 2023OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understandingcited by 184
  10. 2016Thingi10K: A Dataset of 10,000 3D-Printing Modelscited by 183
  11. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 183
  12. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 170
  13. 2017Deep Functional Maps: Structured Prediction for Dense Shape Correspondencecited by 134
  14. 2019SOSNet: Second Order Similarity Regularization for Local Descriptor Learningcited by 131
  15. 2020Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondencecited by 125
  16. 2019PifPaf: Composite Fields for Human Pose Estimationcited by 120
  17. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  18. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 118
  19. 2021Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networkscited by 103
  20. 2016Learning shape correspondence with anisotropic convolutional neural networkscited by 95
  21. 2019BAE-NET: Branched Autoencoder for Shape Co-Segmentationcited by 89
  22. 2020Articulation-aware Canonical Surface Mappingcited by 81
  23. 2020Continuous Surface Embeddingscited by 80
  24. 2023Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmentercited by 76
  25. 2020Deep Shells: Unsupervised Shape Correspondence with Optimal Transportcited by 74
  26. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 63
  27. 2023SATR: Zero-Shot Semantic Segmentation of 3D Shapescited by 62
  28. 2024Training-Free Open-Vocabulary Segmentation with Offline Diffusion-Augmented Prototype Generationcited by 44
  29. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
  30. 2022ExtrudeNet: Unsupervised Inverse Sketch-and-Extrude for Shape Parsingcited by 38
  31. 2019Beyond Cartesian Representations for Local Descriptorscited by 35
  32. 2020Dual Geometric Graph Network (DG2N) -- Iterative network for deformable shape alignmentcited by 35
  33. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 35
  34. 2023Zero-Shot 3D Shape Correspondencecited by 33
  35. 20223D Highlighter: Localizing Regions on 3D Shapes via Text Descriptionscited by 32
  36. 2015Dense Human Body Correspondences Using Convolutional Networkscited by 26
  37. 2024iSeg: Interactive 3D Segmentation via Interactive Attentioncited by 14
  38. 2021Discovering Relationships between Object Categories via Universal Canonical Mapscited by 13

50 most cited of 55 All 55 (JSON)

Built on by

Not cited yet.