# Learning Semantic Correspondence with Sparse Annotations

arXiv: https://arxiv.org/abs/2208.06974 · HTML: https://mlsanity.com/paper/2208.06974/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 63 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 30 of 46)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [A Comprehensive Overhaul of Feature Distillation](https://mlsanity.com/paper/1904.01866/) (2019)
- [Net2Net: Accelerating Learning via Knowledge Transfer](https://mlsanity.com/paper/1511.05641/) (2015)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences](https://mlsanity.com/paper/1912.05524/) (2019)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [End-to-end weakly-supervised semantic alignment](https://mlsanity.com/paper/1712.06861/) (2017)
- [SFNet: Learning Object-aware Semantic Correspondence](https://mlsanity.com/paper/1904.01810/) (2019)
- [DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision](https://mlsanity.com/paper/2105.06464/) (2021)
- [Proposal Flow](https://mlsanity.com/paper/1511.05065/) (2015)
- [Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Features](https://mlsanity.com/paper/1908.06537/) (2019)
- [Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence](https://mlsanity.com/paper/2005.05207/) (2020)
- [Learning to Compose Hypercolumns for Visual Correspondence](https://mlsanity.com/paper/2007.10587/) (2020)
- [SCNet: Learning Semantic Correspondence](https://mlsanity.com/paper/1705.04043/) (2017)
- [Proposal Flow: Semantic Correspondences from Object Proposals](https://mlsanity.com/paper/1703.07144/) (2017)
- [Rethinking Pseudo Labels for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.00168/) (2021)
- [Dynamic Context Correspondence Network for Semantic Alignment](https://mlsanity.com/paper/1909.03444/) (2019)
- [Convolutional Hough Matching Networks](https://mlsanity.com/paper/2103.16831/) (2021)
- [FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence](https://mlsanity.com/paper/1702.00926/) (2017)

## Built on by (most cited first, 30 of 31)

- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching](https://mlsanity.com/paper/2310.17569/) (2023)
- [Towards Scalable Neural Representation for Diverse Videos](https://mlsanity.com/paper/2303.14124/) (2023)
- [Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps](https://mlsanity.com/paper/2312.13216/) (2023)
- [ASIC: Aligning Sparse in-the-wild Image Collections](https://mlsanity.com/paper/2303.16201/) (2023)
- [MangaNinja: Line Art Colorization with Precise Reference Following](https://mlsanity.com/paper/2501.08332/) (2025)
- [P3-PO: Prescriptive Point Priors for Visuo-Spatial Generalization of Robot Policies](https://mlsanity.com/paper/2412.06784/) (2024)
- [Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence](https://mlsanity.com/paper/2403.11120/) (2024)
- [Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels](https://mlsanity.com/paper/2506.05312/) (2025)
- [MATCHA:Towards Matching Anything](https://mlsanity.com/paper/2501.14945/) (2025)
- [ARDuP: Active Region Video Diffusion for Universal Policies](https://mlsanity.com/paper/2406.13301/) (2024)
- [Stable-SCore: A Stable Registration-based Framework for 3D Shape Correspondence](https://mlsanity.com/paper/2503.21766/) (2025)
- [SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations](https://mlsanity.com/paper/2503.22462/) (2025)
- [TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/2505.06079/) (2025)
- [Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognition](https://mlsanity.com/paper/2508.03695/) (2025)
- [SimSC: A Simple Framework for Semantic Correspondence with Temperature Learning](https://mlsanity.com/paper/2305.02385/) (2023)
- [Efficient Semantic Matching with Hypercolumn Correlation](https://mlsanity.com/paper/2311.04336/) (2023)
- [SpaceJAM: a Lightweight and Regularization-free Method for Fast Joint Alignment of Images](https://mlsanity.com/paper/2407.11850/) (2024)
- [Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence](https://mlsanity.com/paper/2506.08220/) (2025)
- [GECO: Geometrically Consistent Embedding with Lightspeed Inference](https://mlsanity.com/paper/2508.00746/) (2025)
- [What is Point Supervision Worth in Video Instance Segmentation?](https://mlsanity.com/paper/2404.01990/) (2024)
- [UVIS: Unsupervised Video Instance Segmentation](https://mlsanity.com/paper/2406.06908/) (2024)
- [Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform](https://mlsanity.com/paper/2404.11156/) (2024)
- [Semantic Correspondence: Unified Benchmarking and a Strong Baseline](https://mlsanity.com/paper/2505.18060/) (2025)
- [Towards Geometry-Grounded Dense Semantic Matching with VGGT Priors](https://mlsanity.com/paper/2509.21263/) (2025)
- [MARCO: Navigating the Unseen Space of Semantic Correspondence](https://mlsanity.com/paper/2604.18267/) (2026)
- [Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Images](https://mlsanity.com/paper/2311.18540/) (2023)
- [Similarity-Aware Selective State-Space Modeling for Semantic Correspondence](https://mlsanity.com/paper/2509.24318/) (2025)

## Abstract (arXiv)

Finding dense semantic correspondence is a fundamental problem in computer vision, which remains challenging in complex scenes due to background clutter, extreme intra-class variation, and a severe lack of ground truth. In this paper, we aim to address the challenge of label sparsity in semantic correspondence by enriching supervision signals from sparse keypoint annotations. To this end, we first propose a teacher-student learning paradigm for generating dense pseudo-labels and then develop two novel strategies for denoising pseudo-labels. In particular, we use spatial priors around the sparse annotations to suppress the noisy pseudo-labels. In addition, we introduce a loss-driven dynamic label selection strategy for label denoising. We instantiate our paradigm with two variants of learning strategies: a single offline teacher setting, and mutual online teachers setting. Our approach achieves notable improvements on three challenging benchmarks for semantic correspondence and establishes the new state-of-the-art. Project page: https://shuaiyihuang.github.io/publications/SCorrSAN.