mlsanityData to 7 Oct 2026

Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence

arXiv 2311.17034 ↗First posted 28 Nov 2023cs.CVcited by 85

Not on the map. Why?

Abstract (arXiv)

While pre-trained large-scale vision models have shown significant promise for semantic correspondence, their features often struggle to grasp the geometry and orientation of instances. This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing. We show that incorporating this information can markedly enhance semantic correspondence performance with simple but effective solutions in both zero-shot and supervised settings. We also construct a new challenging benchmark for semantic correspondence built from an existing animal pose estimation dataset, for both pre-training validating models. Our method achieves a [email protected] score of 65.4 (zero-shot) and 85.6 (supervised) on the challenging SPair-71k dataset, outperforming the state of the art by 5.5p and 11.0p absolute gains, respectively. Our code and datasets are publicly available at: https://telling-left-from-right.github.io/.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 44 of its 61 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

44 of 61 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  9. 2023Emergent Correspondence from Image Diffusioncited by 437
  10. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  11. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  12. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondencecited by 297
32 more
  1. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  2. 2023Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondencecited by 211
  3. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  4. 2019SPair-71k: A Large-scale Benchmark for Semantic Correspondencecited by 140
  5. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  6. 2017Convolutional neural network architecture for geometric matchingcited by 138
  7. 2023Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusioncited by 131
  8. 2021AP-10K: A Benchmark for Animal Pose Estimation in the Wildcited by 123
  9. 2023Unsupervised Semantic Correspondence Using Stable Diffusioncited by 120
  10. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  11. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  12. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  13. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  14. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  15. 2017End-to-end weakly-supervised semantic alignmentcited by 60
  16. 2019SFNet: Learning Object-aware Semantic Correspondencecited by 60
  17. 2015Proposal Flowcited by 57
  18. 2020Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondencecited by 53
  19. 2020Learning to Compose Hypercolumns for Visual Correspondencecited by 53
  20. 2020GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Networkcited by 53
  21. 2022CATs++: Boosting Cost Aggregation with Convolutions and Transformerscited by 53
  22. 2017Proposal Flow: Semantic Correspondences from Object Proposalscited by 46
  23. 2021Warp Consistency for Unsupervised Learning of Dense Correspondencescited by 42
  24. 2018Recurrent Transformer Networks for Semantic Correspondencecited by 32
  25. 2018Attentive Semantic Alignment with Offset-Aware Correlation Kernelscited by 31
  26. 2022Learning Semantic Correspondence with Sparse Annotationscited by 31
  27. 2022Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondencescited by 30
  28. 2023Neural Congealing: Aligning Images to a Joint Semantic Atlascited by 30
  29. 2023ASIC: Aligning Sparse in-the-wild Image Collectionscited by 26
  30. 2019Semantic Attribute Matching Networkscited by 20
  31. 2021A Novel Dataset for Keypoint Detection of quadruped Animals from Imagescited by 18
  32. 2022Demystifying Unsupervised Semantic Correspondence Estimationcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 18; 2025: 42; 2026: 24.

85 papers.

Sort:

By area:

  1. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  2. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  3. 2024RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulationcited by 55
  4. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  5. 2024CleanDIFT: Diffusion Features without Noisecited by 42
  6. 2025AnyTop: Character Animation Diffusion with Any Topologycited by 37
  7. 2023Improving Semantic Correspondence with Viewpoint-Guided Spherical Mapscited by 33
  8. 2025Emergent Temporal Correspondences from Video Diffusion Transformerscited by 30
  9. 2024Distillation of Diffusion Features for Semantic Correspondencecited by 17
  10. 2024Law of Vision Representation in MLLMscited by 16
  11. 2024EffoVPR: Effective Foundation Model Utilization for Visual Place Recognitioncited by 15
  12. 2024Can Visual Foundation Models Achieve Long-term Point Tracking?cited by 14
38 more
  1. 2024Toward a Holistic Evaluation of Robustness in CLIP Modelscited by 13
  2. 2025Do It Yourself: Learning Semantic Correspondence from Pseudo-Labelscited by 13
  3. 2025MOSAIC: Multi-Subject Personalized Generation via Correspondence-Aware Alignment and Disentanglementcited by 13
  4. 2025MATCHA:Towards Matching Anythingcited by 12
  5. 2026INSID3: Training-Free In-Context Segmentation with DINOv3cited by 12
  6. 2024SHIC: Shape-Image Correspondences with no Keypoint Supervisioncited by 11
  7. 2024Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptorscited by 10
  8. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  9. 2025RecDreamer: Consistent Text-to-3D Generation via Uniform Score Distillationcited by 9
  10. 2025Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Spacecited by 9
  11. 2024Eye-for-an-eye: Appearance Transfer with Semantic Correspondence in Diffusion Modelscited by 8
  12. 2025JOG3R: Towards 3D-Consistent Video Generatorscited by 8
  13. 2025CorrFill: Enhancing Faithfulness in Reference-based Inpainting with Correspondence Guidance in Diffusion Modelscited by 8
  14. 2025SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentationcited by 8
  15. 2025Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondencecited by 7
  16. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6
  17. 2025SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representationscited by 6
  18. 2025GECO: Geometrically Consistent Embedding with Lightspeed Inferencecited by 6
  19. 2025Mind-the-Glitch: Visual Correspondence for Detecting Inconsistencies in Subject-Driven Generationcited by 6
  20. 2025Surface-Aware Distilled 3D Semantic Featurescited by 5
  21. 2025Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Modelscited by 5
  22. 2025Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondencecited by 5
  23. 2025Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolutioncited by 5
  24. 2024Diffusion Features for Zero-Shot 6DoF Object Pose Estimationcited by 4
  25. 2025SketchFusion: Learning Universal Sketch Features through Fusing Foundation Modelscited by 4
  26. 2025Learning 3D Scene Analogies with Neural Contextual Scene Mapscited by 4
  27. 2025Studying Image Diffusion Features for Zero-Shot Video Object Segmentationcited by 4
  28. 2025ProTracker: Probabilistic Integration for Robust and Accurate Point Trackingcited by 3
  29. 2025Semantic Correspondence: Unified Benchmarking and a Strong Baselinecited by 3
  30. 2025Symmetry Understanding of 3D Shapes via Chirality Disentanglementcited by 3
  31. 2025Towards Geometry-Grounded Dense Semantic Matching with VGGT Priorscited by 3
  32. 2025Probing and Leveraging Video Diffusion Transformer Features for Robust Point Trackingcited by 3
  33. 2026MARCO: Navigating the Unseen Space of Semantic Correspondencecited by 3
  34. 2024TrafficLoc: Localizing Traffic Surveillance Cameras in 3D Scenescited by 2
  35. 2025ViT-VS: On the Applicability of Pretrained Vision Transformer Features for Generalizable Visual Servoingcited by 2
  36. 2025Self-Supervised Spatial Correspondence Across Modalitiescited by 2
  37. 2025Multi-View Foundation Modelscited by 2
  38. 2026Symmetry Informative and Agnostic Feature Disentanglement for 3D Shapescited by 2

50 most cited of 85 All 85 (JSON)