# Learning Token-based Representation for Image Retrieval

arXiv: https://arxiv.org/abs/2112.06159 · HTML: https://mlsanity.com/paper/2112.06159/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2021.

## Built on by (most cited first, 10 of 10)

- [DisEnvisioner: Disentangled and Enriched Visual Prompt for Customized Image Generation](https://mlsanity.com/paper/2410.02067/) (2024)
- [Graph Convolution Based Efficient Re-Ranking for Visual Retrieval](https://mlsanity.com/paper/2306.08792/) (2023)
- [Coarse-to-Fine: Learning Compact Discriminative Representation for Single-Stage Image Retrieval](https://mlsanity.com/paper/2308.04008/) (2023)
- [On Train-Test Class Overlap and Detection for Image Retrieval](https://mlsanity.com/paper/2404.01524/) (2024)
- [Region-aware Image-based Human Action Retrieval with Transformers](https://mlsanity.com/paper/2407.09924/) (2024)
- [Efficient Image-Text Retrieval via Keyword-Guided Pre-Screening](https://mlsanity.com/paper/2303.07740/) (2023)
- [Retrieval Robust to Object Motion Blur](https://mlsanity.com/paper/2404.18025/) (2024)
- [AdapNet: Adaptive Noise-Based Network for Low-Quality Image Retrieval](https://mlsanity.com/paper/2405.17718/) (2024)
- [Accurate and Fast Pixel Retrieval with Spatial and Uncertainty Aware Hypergraph Diffusion](https://mlsanity.com/paper/2406.11242/) (2024)
- [KnifeHunter: Structured Local Representation Learning for Fine-Grained Knife Image Retrieval in Law Enforcement](https://mlsanity.com/paper/2608.07057/) (2026)

## Abstract (arXiv)

In image retrieval, deep local features learned in a data-driven manner have been demonstrated effective to improve retrieval performance. To realize efficient retrieval on large image database, some approaches quantize deep local features with a large codebook and match images with aggregated match kernel. However, the complexity of these approaches is non-trivial with large memory footprint, which limits their capability to jointly perform feature learning and aggregation. To generate compact global representations while maintaining regional matching capability, we propose a unified framework to jointly learn local feature representation and aggregation. In our framework, we first extract deep local features using CNNs. Then, we design a tokenizer module to aggregate them into a few visual tokens, each corresponding to a specific visual pattern. This helps to remove background noise, and capture more discriminative regions in the image. Next, a refinement block is introduced to enhance the visual tokens with self-attention and cross-attention. Finally, different visual tokens are concatenated to generate a compact global representation. The whole framework is trained end-to-end with image-level labels. Extensive experiments are conducted to evaluate our approach, which outperforms the state-of-the-art methods on the Revisited Oxford and Paris datasets.