# SFNet: Learning Object-aware Semantic Correspondence

arXiv: https://arxiv.org/abs/1904.01810 · HTML: https://mlsanity.com/paper/1904.01810/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2019.

## Built on by (most cited first, 30 of 60)

- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [Local All-Pair Correspondence for Point Tracking](https://mlsanity.com/paper/2407.15420/) (2024)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision](https://mlsanity.com/paper/2105.06464/) (2021)
- [Zero-Shot Category-Level Object Pose Estimation](https://mlsanity.com/paper/2204.03635/) (2022)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [CATs++: Boosting Cost Aggregation with Convolutions and Transformers](https://mlsanity.com/paper/2202.06817/) (2022)
- [Going Denser with Open-Vocabulary Part Segmentation](https://mlsanity.com/paper/2305.11173/) (2023)
- [Convolutional Hough Matching Networks](https://mlsanity.com/paper/2103.16831/) (2021)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [Visual correspondence-based explanations improve AI robustness and human-AI team accuracy](https://mlsanity.com/paper/2208.00780/) (2022)
- [SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching](https://mlsanity.com/paper/2310.17569/) (2023)
- [Multi-scale Matching Networks for Semantic Correspondence](https://mlsanity.com/paper/2108.00211/) (2021)
- [Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence](https://mlsanity.com/paper/2210.02689/) (2022)
- [Learning Semantic Correspondence with Sparse Annotations](https://mlsanity.com/paper/2208.06974/) (2022)
- [Deep Matching Prior: Test-Time Optimization for Dense Correspondence](https://mlsanity.com/paper/2106.03090/) (2021)
- [Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences](https://mlsanity.com/paper/2203.04279/) (2022)
- [Cross-View Completion Models are Zero-shot Correspondence Estimators](https://mlsanity.com/paper/2412.09072/) (2024)
- [TransforMatcher: Match-to-Match Attention for Semantic Correspondence](https://mlsanity.com/paper/2205.11634/) (2022)
- [ASIC: Aligning Sparse in-the-wild Image Collections](https://mlsanity.com/paper/2303.16201/) (2023)
- [Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style](https://mlsanity.com/paper/2303.14348/) (2023)
- [MV-TON: Memory-based Video Virtual Try-on network](https://mlsanity.com/paper/2108.07502/) (2021)
- [FDAN: Flow-guided Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/2105.05640/) (2021)
- [Distance-aware Quantization](https://mlsanity.com/paper/2108.06983/) (2021)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [Cost Aggregation Is All You Need for Few-Shot Segmentation](https://mlsanity.com/paper/2112.11685/) (2021)
- [Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels](https://mlsanity.com/paper/2203.16038/) (2022)

## Abstract (arXiv)

We address the problem of semantic correspondence, that is, establishing a dense flow field between images depicting different instances of the same object or scene category. We propose to use images annotated with binary foreground masks and subjected to synthetic geometric deformations to train a convolutional neural network (CNN) for this task. Using these masks as part of the supervisory signal offers a good compromise between semantic flow methods, where the amount of training data is limited by the cost of manually selecting point correspondences, and semantic alignment ones, where the regression of a single global geometric transformation between images may be sensitive to image-specific details such as background clutter. We propose a new CNN architecture, dubbed SFNet, which implements this idea. It leverages a new and differentiable version of the argmax function for end-to-end training, with a loss that combines mask and flow consistency with smoothness terms. Experimental results demonstrate the effectiveness of our approach, which significantly outperforms the state of the art on standard benchmarks.