# Independently Keypoint Learning for Small Object Semantic Correspondence

arXiv: https://arxiv.org/abs/2404.02678 · HTML: https://mlsanity.com/paper/2404.02678/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 57 references that are in the map.
- First posted on arXiv 3 Apr 2024.

## Builds on (most cited first, 30 of 42)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [Context Encoding for Semantic Segmentation](https://mlsanity.com/paper/1803.08904/) (2018)
- [UnitBox: An Advanced Object Detection Network](https://mlsanity.com/paper/1608.01471/) (2016)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [COTR: Correspondence Transformer for Matching Across Images](https://mlsanity.com/paper/2103.14167/) (2021)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.13132/) (2022)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [UIU-Net: U-Net in U-Net for Infrared Small Object Detection](https://mlsanity.com/paper/2212.00968/) (2022)
- [Fast-SCNN: Fast Semantic Segmentation Network](https://mlsanity.com/paper/1902.04502/) (2019)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [M2Det: A Single-Shot Object Detector based on Multi-Level Feature Pyramid Network](https://mlsanity.com/paper/1811.04533/) (2018)
- [Semantic Segmentation using Vision Transformers: A survey](https://mlsanity.com/paper/2305.03273/) (2023)
- [DGC-Net: Dense Geometric Correspondence Network](https://mlsanity.com/paper/1810.08393/) (2018)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [Proposal Flow](https://mlsanity.com/paper/1511.05065/) (2015)
- [Hyperpixel Flow: Semantic Correspondence with Multi-layer Neural Features](https://mlsanity.com/paper/1908.06537/) (2019)
- [Learning to Compose Hypercolumns for Visual Correspondence](https://mlsanity.com/paper/2007.10587/) (2020)
- [SCNet: Learning Semantic Correspondence](https://mlsanity.com/paper/1705.04043/) (2017)

## Built on by (most cited first, 2 of 2)

- [Semantic Correspondence: Unified Benchmarking and a Strong Baseline](https://mlsanity.com/paper/2505.18060/) (2025)
- [SimpleMatch: A Simple and Strong Baseline for Semantic Correspondence](https://mlsanity.com/paper/2601.12357/) (2026)

## Abstract (arXiv)

Semantic correspondence remains a challenging task for establishing correspondences between a pair of images with the same category or similar scenes due to the large intra-class appearance. In this paper, we introduce a novel problem called 'Small Object Semantic Correspondence (SOSC).' This problem is challenging due to the close proximity of keypoints associated with small objects, which results in the fusion of these respective features. It is difficult to identify the corresponding key points of the fused features, and it is also difficult to be recognized. To address this challenge, we propose the Keypoint Bounding box-centered Cropping (KBC) method, which aims to increase the spatial separation between keypoints of small objects, thereby facilitating independent learning of these keypoints. The KBC method is seamlessly integrated into our proposed inference pipeline and can be easily incorporated into other methodologies, resulting in significant performance enhancements. Additionally, we introduce a novel framework, named KBCNet, which serves as our baseline model. KBCNet comprises a Cross-Scale Feature Alignment (CSFA) module and an efficient 4D convolutional decoder. The CSFA module is designed to align multi-scale features, enriching keypoint representations by integrating fine-grained features and deep semantic features. Meanwhile, the 4D convolutional decoder, based on efficient 4D convolution, ensures efficiency and rapid convergence. To empirically validate the effectiveness of our proposed methodology, extensive experiments are conducted on three widely used benchmarks: PF-PASCAL, PF-WILLOW, and SPair-71k. Our KBC method demonstrates a substantial performance improvement of 7.5% on the SPair-71K dataset, providing compelling evidence of its efficacy.