# AMES: Asymmetric and Memory-Efficient Similarity Estimation for Instance-level Retrieval

arXiv: https://arxiv.org/abs/2408.03282 · HTML: https://mlsanity.com/paper/2408.03282/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 74 references that are in the map.
- First posted on arXiv 6 Aug 2024.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [FlexiViT: One Model for All Patch Sizes](https://mlsanity.com/paper/2212.08013/) (2022)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [Randomized Positional Encodings Boost Length Generalization of Transformers](https://mlsanity.com/paper/2305.16843/) (2023)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [Revisiting Training Strategies and Generalization Performance in Deep Metric Learning](https://mlsanity.com/paper/2002.08473/) (2020)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [Learnable pooling with Context Gating for video classification](https://mlsanity.com/paper/1706.06905/) (2017)
- [Learning and aggregating deep local descriptors for instance-level recognition](https://mlsanity.com/paper/2007.13172/) (2020)

## Built on by (most cited first, 16 of 16)

- [ILIAS: Instance-Level Image retrieval At Scale](https://mlsanity.com/paper/2502.11748/) (2025)
- [LOCORE: Image Re-ranking with Long-Context Sequence Modeling](https://mlsanity.com/paper/2503.21772/) (2025)
- [Weakly Supervised Temporal Sentence Grounding via Positive Sample Mining](https://mlsanity.com/paper/2505.06557/) (2025)
- [QuARI: Query Adaptive Retrieval Improvement](https://mlsanity.com/paper/2505.21647/) (2025)
- [VLM-Guided Visual Place Recognition for Planet-Scale Geo-Localization](https://mlsanity.com/paper/2507.17455/) (2025)
- [Global-to-Local or Local-to-Global? Enhancing Image Retrieval with Efficient Local Search and Effective Global Re-ranking](https://mlsanity.com/paper/2509.04351/) (2025)
- [Patch-wise Retrieval: A Bag of Practical Techniques for Instance-level Matching](https://mlsanity.com/paper/2512.12610/) (2025)
- [Find your Needle: Small Object Image Retrieval via Multi-Object Attention Optimization](https://mlsanity.com/paper/2503.07038/) (2025)
- [Instance-Level Generation for Representation Learning](https://mlsanity.com/paper/2510.09171/) (2025)
- [ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains](https://mlsanity.com/paper/2603.28603/) (2026)
- [Indexing Multimodal Language Models for Large-scale Image Retrieval](https://mlsanity.com/paper/2604.13268/) (2026)
- [Tracking Phenological Status and Ecological Interactions in a Hawaiian Cloud Forest Understory using Low-Cost Camera Traps and Visual Foundation Models](https://mlsanity.com/paper/2603.07817/) (2026)
- [AsymLoc: Towards Asymmetric Feature Matching for Efficient Visual Localization](https://mlsanity.com/paper/2604.09445/) (2026)
- [KnifeHunter: Structured Local Representation Learning for Fine-Grained Knife Image Retrieval in Law Enforcement](https://mlsanity.com/paper/2608.07057/) (2026)
- [FoundYou: A Unified Model for Personalized Segmentation and Retrieval](https://mlsanity.com/paper/2608.29917/) (2026)
- [When Does Geometric View Synthesis Help Wine Label Retrieval? A Public One-Shot Benchmark Across Self-Supervised and Vision-Language Backbones](https://mlsanity.com/paper/2609.33359/) (2026)

## Abstract (arXiv)

This work investigates the problem of instance-level image retrieval re-ranking with the constraint of memory efficiency, ultimately aiming to limit memory usage to 1KB per image. Departing from the prevalent focus on performance enhancements, this work prioritizes the crucial trade-off between performance and memory requirements. The proposed model uses a transformer-based architecture designed to estimate image-to-image similarity by capturing interactions within and across images based on their local descriptors. A distinctive property of the model is the capability for asymmetric similarity estimation. Database images are represented with a smaller number of descriptors compared to query images, enabling performance improvements without increasing memory consumption. To ensure adaptability across different applications, a universal model is introduced that adjusts to a varying number of local descriptors during the testing phase. Results on standard benchmarks demonstrate the superiority of our approach over both hand-crafted and learned models. In particular, compared with current state-of-the-art methods that overlook their memory footprint, our approach not only attains superior performance but does so with a significantly reduced memory footprint. The code and pretrained models are publicly available at: https://github.com/pavelsuma/ames