# Joint Learning of Feature Extraction and Cost Aggregation for Semantic Correspondence

arXiv: https://arxiv.org/abs/2204.02164 · HTML: https://mlsanity.com/paper/2204.02164/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss](https://mlsanity.com/paper/1711.07837/) (2017)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [Learning Feature Descriptors using Camera Pose Supervision](https://mlsanity.com/paper/2004.13324/) (2020)
- [Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions](https://mlsanity.com/paper/2004.10566/) (2020)
- [Universal Correspondence Network](https://mlsanity.com/paper/1606.03558/) (2016)
- [DGC-Net: Dense Geometric Correspondence Network](https://mlsanity.com/paper/1810.08393/) (2018)
- [DDFlow: Learning Optical Flow with Unlabeled Data Distillation](https://mlsanity.com/paper/1902.09145/) (2019)
- [End-to-end weakly-supervised semantic alignment](https://mlsanity.com/paper/1712.06861/) (2017)
- [SFNet: Learning Object-aware Semantic Correspondence](https://mlsanity.com/paper/1904.01810/) (2019)
- [Proposal Flow](https://mlsanity.com/paper/1511.05065/) (2015)
- [Dynamic Context Correspondence Network for Semantic Alignment](https://mlsanity.com/paper/1909.03444/) (2019)
- [FCSS: Fully Convolutional Self-Similarity for Dense Semantic Correspondence](https://mlsanity.com/paper/1702.00926/) (2017)
- [Correspondence Networks with Adaptive Neighbourhood Consensus](https://mlsanity.com/paper/2003.12059/) (2020)
- [Recurrent Transformer Networks for Semantic Correspondence](https://mlsanity.com/paper/1810.12155/) (2018)
- [Is This The Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization](https://mlsanity.com/paper/1908.04598/) (2019)

## Abstract (arXiv)

Establishing dense correspondences across semantically similar images is one of the challenging tasks due to the significant intra-class variations and background clutters. To solve these problems, numerous methods have been proposed, focused on learning feature extractor or cost aggregation independently, which yields sub-optimal performance. In this paper, we propose a novel framework for jointly learning feature extraction and cost aggregation for semantic correspondence. By exploiting the pseudo labels from each module, the networks consisting of feature extraction and cost aggregation modules are simultaneously learned in a boosting fashion. Moreover, to ignore unreliable pseudo labels, we present a confidence-aware contrastive loss function for learning the networks in a weakly-supervised manner. We demonstrate our competitive results on standard benchmarks for semantic correspondence.