# Learning Dense Correspondence via 3D-guided Cycle Consistency

arXiv: https://arxiv.org/abs/1604.05383 · HTML: https://mlsanity.com/paper/1604.05383/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2016.

## Built on by (most cited first, 30 of 91)

- [Tracking Everything Everywhere All at Once](https://mlsanity.com/paper/2306.05422/) (2023)
- [Taming Visually Guided Sound Generation](https://mlsanity.com/paper/2110.08791/) (2021)
- [Few-Shot Segmentation via Cycle-Consistent Transformer](https://mlsanity.com/paper/2106.02320/) (2021)
- [Third-Person Visual Imitation Learning via Decoupled Hierarchical Controller](https://mlsanity.com/paper/1911.09676/) (2019)
- [DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video](https://mlsanity.com/paper/2403.14548/) (2024)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning](https://mlsanity.com/paper/2106.04152/) (2021)
- [DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generation](https://mlsanity.com/paper/2503.15208/) (2025)
- [Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion](https://mlsanity.com/paper/2103.16690/) (2021)
- [TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolation](https://mlsanity.com/paper/2203.13859/) (2022)
- [High-Fidelity and Arbitrary Face Editing](https://mlsanity.com/paper/2103.15814/) (2021)
- [Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps](https://mlsanity.com/paper/2312.13216/) (2023)
- [Geometric Unsupervised Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2103.16694/) (2021)
- [Learning Optical Flow, Depth, and Scene Flow without Real-World Labels](https://mlsanity.com/paper/2203.15089/) (2022)
- [Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences](https://mlsanity.com/paper/2203.04279/) (2022)
- [Virtual Correspondence: Humans as a Cue for Extreme-View Geometry](https://mlsanity.com/paper/2206.08365/) (2022)
- [Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild](https://mlsanity.com/paper/2210.07199/) (2022)
- [Mask Matching Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2301.01208/) (2022)
- [Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferences](https://mlsanity.com/paper/2506.02095/) (2025)
- [Learning 3D Dense Correspondence via Canonical Point Autoencoder](https://mlsanity.com/paper/2107.04867/) (2021)
- [Bi-level Feature Alignment for Versatile Image Translation and Manipulation](https://mlsanity.com/paper/2107.03021/) (2021)
- [Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency](https://mlsanity.com/paper/2105.06463/) (2021)
- [MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Frames](https://mlsanity.com/paper/2102.12841/) (2021)
- [Learning monocular 3D reconstruction of articulated categories from motion](https://mlsanity.com/paper/2103.16352/) (2021)
- [Object Permanence Emerges in a Random Walk along Memory](https://mlsanity.com/paper/2204.01784/) (2022)
- [Deep Co-Attention Network for Multi-View Subspace Learning](https://mlsanity.com/paper/2102.07751/) (2021)
- [Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles](https://mlsanity.com/paper/2103.14098/) (2021)
- [Consistency Learning via Decoding Path Augmentation for Transformers in Human Object Interaction Detection](https://mlsanity.com/paper/2204.04836/) (2022)
- [UDoc-GAN: Unpaired Document Illumination Correction with Background Light Prior](https://mlsanity.com/paper/2210.08216/) (2022)
- [Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence](https://mlsanity.com/paper/2403.11120/) (2024)

## Abstract (arXiv)

Discriminative deep learning approaches have shown impressive results for problems where human-labeled ground truth is plentiful, but what about tasks where labels are difficult or impossible to obtain? This paper tackles one such problem: establishing dense visual correspondence across different object instances. For this task, although we do not know what the ground-truth is, we know it should be consistent across instances of that category. We exploit this consistency as a supervisory signal to train a convolutional neural network to predict cross-instance correspondences between pairs of images depicting objects of the same category. For each pair of training images we find an appropriate 3D CAD model and render two synthetic views to link in with the pair, establishing a correspondence flow 4-cycle. We use ground-truth synthetic-to-synthetic correspondences, provided by the rendering engine, to train a ConvNet to predict synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent with the ground-truth. At test time, no CAD models are required. We demonstrate that our end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.