# Recurrent Transformer Networks for Semantic Correspondence

arXiv: https://arxiv.org/abs/1810.12155 · HTML: https://mlsanity.com/paper/1810.12155/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2018.

## Built on by (most cited first, 30 of 32)

- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision](https://mlsanity.com/paper/2105.06464/) (2021)
- [CATs++: Boosting Cost Aggregation with Convolutions and Transformers](https://mlsanity.com/paper/2202.06817/) (2022)
- [Convolutional Hough Matching Networks](https://mlsanity.com/paper/2103.16831/) (2021)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [Deep Lucas-Kanade Homography for Multimodal Image Alignment](https://mlsanity.com/paper/2104.11693/) (2021)
- [Multi-scale Matching Networks for Semantic Correspondence](https://mlsanity.com/paper/2108.00211/) (2021)
- [Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence](https://mlsanity.com/paper/2210.02689/) (2022)
- [Learning Semantic Correspondence with Sparse Annotations](https://mlsanity.com/paper/2208.06974/) (2022)
- [Deep Matching Prior: Test-Time Optimization for Dense Correspondence](https://mlsanity.com/paper/2106.03090/) (2021)
- [Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences](https://mlsanity.com/paper/2203.04279/) (2022)
- [Cross-View Completion Models are Zero-shot Correspondence Estimators](https://mlsanity.com/paper/2412.09072/) (2024)
- [TransforMatcher: Match-to-Match Attention for Semantic Correspondence](https://mlsanity.com/paper/2205.11634/) (2022)
- [MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation](https://mlsanity.com/paper/2209.11047/) (2022)
- [FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network](https://mlsanity.com/paper/2301.08160/) (2023)
- [Diffusion Model for Dense Matching](https://mlsanity.com/paper/2305.19094/) (2023)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels](https://mlsanity.com/paper/2203.16038/) (2022)
- [Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence](https://mlsanity.com/paper/2109.05221/) (2021)
- [Learning Contrastive Representation for Semantic Correspondence](https://mlsanity.com/paper/2109.10967/) (2021)
- [Demystifying Unsupervised Semantic Correspondence Estimation](https://mlsanity.com/paper/2207.05054/) (2022)
- [Zero-Shot Image Feature Consensus with Deep Functional Maps](https://mlsanity.com/paper/2403.12038/) (2024)
- [Industrial Style Transfer with Large-scale Geometric Warping and Content Preservation](https://mlsanity.com/paper/2203.12835/) (2022)
- [Integrative Feature and Cost Aggregation with Transformers for Dense Correspondence](https://mlsanity.com/paper/2209.08742/) (2022)
- [Learning 3D Scene Analogies with Neural Contextual Scene Maps](https://mlsanity.com/paper/2503.15897/) (2025)
- [DocAligner: Annotating Real-world Photographic Document Images by Simply Taking Pictures](https://mlsanity.com/paper/2306.05749/) (2023)
- [Semantic Correspondence: Unified Benchmarking and a Strong Baseline](https://mlsanity.com/paper/2505.18060/) (2025)
- [Modeling Object Dissimilarity for Deep Saliency Prediction](https://mlsanity.com/paper/2104.03864/) (2021)

## Abstract (arXiv)

We present recurrent transformer networks (RTNs) for obtaining dense correspondences between semantically similar images. Our networks accomplish this through an iterative process of estimating spatial transformations between the input images and using these transformations to generate aligned convolutional activations. By directly estimating the transformations between an image pair, rather than employing spatial transformer networks to independently normalize each individual image, we show that greater accuracy can be achieved. This process is conducted in a recursive manner to refine both the transformation estimates and the feature representations. In addition, a technique is presented for weakly-supervised training of RTNs that is based on a proposed classification loss. With RTNs, state-of-the-art performance is attained on several benchmarks for semantic correspondence.