# CAT: Cross-Attention Transformer for One-Shot Object Detection

arXiv: https://arxiv.org/abs/2104.14984 · HTML: https://mlsanity.com/paper/2104.14984/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [Few-shot Object Detection via Feature Reweighting](https://mlsanity.com/paper/1812.01866/) (2018)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector](https://mlsanity.com/paper/1908.01998/) (2019)
- [RepMet: Representative-based metric learning for classification and one-shot object detection](https://mlsanity.com/paper/1806.04728/) (2018)
- [LSTD: A Low-Shot Transfer Detector for Object Detection](https://mlsanity.com/paper/1803.01529/) (2018)
- [One-Shot Object Detection with Co-Attention and Co-Excitation](https://mlsanity.com/paper/1911.12529/) (2019)
- [One-Shot Instance Segmentation](https://mlsanity.com/paper/1811.11507/) (2018)
- [OS2D: One-Stage One-Shot Object Detection by Matching Anchor Features](https://mlsanity.com/paper/2003.06800/) (2020)

## Built on by (most cited first, 3 of 3)

- [Affordance Grounding from Demonstration Video to Target Image](https://mlsanity.com/paper/2303.14644/) (2023)
- [Semantic-aligned Fusion Transformer for One-shot Object Detection](https://mlsanity.com/paper/2203.09093/) (2022)
- [A Survey of Deep Learning for Low-Shot Object Detection](https://mlsanity.com/paper/2112.02814/) (2021)

## Abstract (arXiv)

Given a query patch from a novel class, one-shot object detection aims to detect all instances of that class in a target image through the semantic similarity comparison. However, due to the extremely limited guidance in the novel class as well as the unseen appearance difference between query and target instances, it is difficult to appropriately exploit their semantic similarity and generalize well. To mitigate this problem, we present a universal Cross-Attention Transformer (CAT) module for accurate and efficient semantic similarity comparison in one-shot object detection. The proposed CAT utilizes transformer mechanism to comprehensively capture bi-directional correspondence between any paired pixels from the query and the target image, which empowers us to sufficiently exploit their semantic characteristics for accurate similarity comparison. In addition, the proposed CAT enables feature dimensionality compression for inference speedup without performance loss. Extensive experiments on COCO, VOC, and FSOD under one-shot settings demonstrate the effectiveness and efficiency of our method, e.g., it surpasses CoAE, a major baseline in this task by 1.0% in AP on COCO and runs nearly 2.5 times faster. Code will be available in the future.