# Vision Transformer Hashing for Image Retrieval

arXiv: https://arxiv.org/abs/2109.12564 · HTML: https://mlsanity.com/paper/2109.12564/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 26 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [HOTR: End-to-End Human-Object Interaction Detection with Transformers](https://mlsanity.com/paper/2104.13682/) (2021)
- [HashNet: Deep Learning to Hash by Continuation](https://mlsanity.com/paper/1702.00758/) (2017)
- [Diverse Part Discovery: Occluded Person Re-identification with Part-Aware Transformer](https://mlsanity.com/paper/2106.04095/) (2021)
- [Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers](https://mlsanity.com/paper/2103.16553/) (2021)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Central Similarity Quantization for Efficient Image and Video Retrieval](https://mlsanity.com/paper/1908.00347/) (2019)
- [A Decade Survey of Content Based Image Retrieval using Deep Learning](https://mlsanity.com/paper/2012.00641/) (2020)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [Medical Image Segmentation Using Squeeze-and-Expansion Transformers](https://mlsanity.com/paper/2105.09511/) (2021)
- [Subtype-aware Unsupervised Domain Adaptation for Medical Diagnosis](https://mlsanity.com/paper/2101.00318/) (2021)
- [Investigating the Vision Transformer Model for Image Retrieval Tasks](https://mlsanity.com/paper/2101.03771/) (2021)
- [TransHash: Transformer-based Hamming Hashing for Efficient Image Retrieval](https://mlsanity.com/paper/2105.01823/) (2021)
- [Push for Quantization: Deep Fisher Hashing](https://mlsanity.com/paper/1909.00206/) (2019)

## Built on by (most cited first, 9 of 9)

- [Retrieval-Augmented Transformer for Image Captioning](https://mlsanity.com/paper/2207.13162/) (2022)
- [Towards Retrieval-Augmented Architectures for Image Captioning](https://mlsanity.com/paper/2405.13127/) (2024)
- [Group Generalized Mean Pooling for Vision Transformer](https://mlsanity.com/paper/2212.04114/) (2022)
- [Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos](https://mlsanity.com/paper/2112.08117/) (2021)
- [Context Unaware Knowledge Distillation for Image Retrieval](https://mlsanity.com/paper/2207.09070/) (2022)
- [Weighted Contrastive Hashing](https://mlsanity.com/paper/2209.14099/) (2022)
- [A Novel Site-Agnostic Multimodal Deep Learning Model to Identify Pro-Eating Disorder Content on Social Media](https://mlsanity.com/paper/2307.06775/) (2023)
- [Deep supervised hashing for fast retrieval of radio image cubes](https://mlsanity.com/paper/2309.00932/) (2023)
- [PTSR: Patch Translator for Image Super-Resolution](https://mlsanity.com/paper/2310.13216/) (2023)

## Abstract (arXiv)

Deep learning has shown a tremendous growth in hashing techniques for image retrieval. Recently, Transformer has emerged as a new architecture by utilizing self-attention without convolution. Transformer is also extended to Vision Transformer (ViT) for the visual recognition with a promising performance on ImageNet. In this paper, we propose a Vision Transformer based Hashing (VTS) for image retrieval. We utilize the pre-trained ViT on ImageNet as the backbone network and add the hashing head. The proposed VTS model is fine tuned for hashing under six different image retrieval frameworks, including Deep Supervised Hashing (DSH), HashNet, GreedyHash, Improved Deep Hashing Network (IDHN), Deep Polarized Network (DPN) and Central Similarity Quantization (CSQ) with their objective functions. We perform the extensive experiments on CIFAR10, ImageNet, NUS-Wide, and COCO datasets. The proposed VTS based image retrieval outperforms the recent state-of-the-art hashing techniques with a great margin. We also find the proposed VTS model as the backbone network is better than the existing networks, such as AlexNet and ResNet. The code is released at urlhttps://github.com/shivram1987/VisionTransformerHashing.