mlsanityData to 7 Oct 2026

SHIC: Shape-Image Correspondences with no Keypoint Supervision

arXiv 2407.18907 ↗First posted 26 Jul 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Canonical surface mapping generalizes keypoint detection by assigning each pixel of an object to a corresponding point in a 3D template. Popularised by DensePose for the analysis of humans, authors have since attempted to apply the concept to more categories, but with limited success due to the high cost of manual supervision. In this work, we introduce SHIC, a method to learn canonical maps without manual supervision which achieves better results than supervised methods for most categories. Our idea is to leverage foundation computer vision models such as DINO and Stable Diffusion that are open-ended and thus possess excellent priors over natural categories. SHIC reduces the problem of estimating image-to-template correspondences to predicting image-to-image correspondences using features from the foundation models. The reduction works by matching images of the object to non-photorealistic renders of the template, which emulates the process of collecting manual annotations for this task. These correspondences are then used to supervise high-quality canonical maps for any object of interest. We also show that image generators can further improve the realism of the template views, which provide an additional source of supervision for the model.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 45 of its 54 references that are in the map. First posted on arXiv 26 Jul 2024.

Builds on

45 of 54 references.

References · 45

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  8. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  9. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  10. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  11. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  12. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
33 more
  1. 2023Emergent Correspondence from Image Diffusioncited by 437
  2. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  3. 2016Convolutional Pose Machinescited by 325
  4. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  5. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  6. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
  7. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  8. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 211
  9. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  10. 2017Convolutional neural network architecture for geometric matchingcited by 138
  11. 2019PifPaf: Composite Fields for Human Pose Estimationcited by 120
  12. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  13. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  14. 2014Part-based R-CNNs for Fine-grained Category Detectioncited by 107
  15. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  16. 2023Telling Left from Right: Identifying Geometry-Aware Semantic Correspondencecited by 85
  17. 2020Articulation-aware Canonical Surface Mappingcited by 80
  18. 2020Continuous Surface Embeddingscited by 79
  19. 2018DGC-Net: Dense Geometric Correspondence Networkcited by 67
  20. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 62
  21. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  22. 2015Proposal Flowcited by 57
  23. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  24. 2020GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Networkcited by 53
  25. 2017Proposal Flow: Semantic Correspondences from Object Proposalscited by 46
  26. 2021Warp Consistency for Unsupervised Learning of Dense Correspondencescited by 42
  27. 2023SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matchingcited by 38
  28. 2022Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondencescited by 30
  29. 2018PARN: Pyramidal Affine Regression Networks for Dense Semantic Correspondencecited by 26
  30. 2016WarpNet: Weakly Supervised Matching for Single-view Reconstructioncited by 22
  31. 2021Discovering Relationships between Object Categories via Universal Canonical Mapscited by 13
  32. 2023Neural Semantic Surface Mapscited by 13
  33. 20233D-MuPPET: 3D Multi-Pigeon Pose Estimation and Trackingcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  2. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  3. 2025Surface-Aware Distilled 3D Semantic Featurescited by 5
  4. 2026Symmetry Informative and Agnostic Feature Disentanglement for 3D Shapescited by 2
  5. 2026MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videoscited by 1
  6. 2026Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondencecited by 1
  7. 2025DINeMo: Learning Neural Mesh Models with no 3D Annotationscited by 0
  8. 2026Shape-of-You: Fused Gromov-Wasserstein Optimal Transport for Semantic Correspondence in-the-Wildcited by 0
  9. 2026Best Segmentation Buddies for Image-Shape Correspondencecited by 0
  10. 2026Category-Level 3D Correspondence in Camera Space via Morphable Object Priorscited by 0
  11. 2026Emergence of a Shared Canonical Object Frame from In-the-Wild Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond 'Templates': Category-Agnostic Object Pose, Size, and Shape Estimation from a Single View
  2. 2025Weakly-Supervised Learning of Dense Functional Correspondences
  3. 2022MvDeCor: Multi-view Dense Correspondence Learning for Fine-grained 3D Segmentation
  4. 2024ZeroKey: Point-Level Reasoning and Zero-Shot 3D Keypoint Detection from Large Language Models
  5. 2026SGMatch: Semantic-Guided Non-Rigid Shape Matching with Flow Regularization
  6. 2025KeyPointDiffuser: Unsupervised 3D Keypoint Learning via Latent Diffusion Models
  7. 2026Prompting Image Generators for Training-free Primitive Shape Abstraction
  8. 2024SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation