# STIR: Siamese Transformer for Image Retrieval Postprocessing

arXiv: https://arxiv.org/abs/2304.13393 · HTML: https://mlsanity.com/paper/2304.13393/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Hyperbolic Vision Transformers: Combining Improvements in Metric Learning](https://mlsanity.com/paper/2203.10833/) (2022)
- [ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis](https://mlsanity.com/paper/2004.01113/) (2020)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [Local Features and Visual Words Emerge in Activations](https://mlsanity.com/paper/1905.06358/) (2019)
- [In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation](https://mlsanity.com/paper/1912.07863/) (2019)
- [It Takes Two to Tango: Mixup for Deep Metric Learning](https://mlsanity.com/paper/2106.04990/) (2021)
- [Combination of Multiple Global Descriptors for Image Retrieval](https://mlsanity.com/paper/1903.10663/) (2019)
- [Robust and Decomposable Average Precision for Image Retrieval](https://mlsanity.com/paper/2110.01445/) (2021)
- [Hierarchical Average Precision Training for Pertinent Image Retrieval](https://mlsanity.com/paper/2207.04873/) (2022)

## Built on by (most cited first, 2 of 2)

- [Cross-Domain Identity Representation for Skull to Face Matching with Benchmark DataSet](https://mlsanity.com/paper/2507.08329/) (2025)
- [MARs: Multi-view Attention Regularizations for Patch-based Feature Recognition of Space Terrain](https://mlsanity.com/paper/2410.05182/) (2024)

## Abstract (arXiv)

Current metric learning approaches for image retrieval are usually based on learning a space of informative latent representations where simple approaches such as the cosine distance will work well. Recent state of the art methods such as HypViT move to more complex embedding spaces that may yield better results but are harder to scale to production environments. In this work, we first construct a simpler model based on triplet loss with hard negatives mining that performs at the state of the art level but does not have these drawbacks. Second, we introduce a novel approach for image retrieval postprocessing called Siamese Transformer for Image Retrieval (STIR) that reranks several top outputs in a single forward pass. Unlike previously proposed Reranking Transformers, STIR does not rely on global/local feature extraction and directly compares a query image and a retrieved candidate on pixel level with the usage of attention mechanism. The resulting approach defines a new state of the art on standard image retrieval datasets: Stanford Online Products and DeepFashion In-shop. We also release the source code at https://github.com/OML-Team/open-metric-learning/tree/main/pipelines/postprocessing/ and an interactive demo of our approach at https://dapladoc-oml-postprocessing-demo-srcappmain-pfh2g0.streamlit.app/