# LOCORE: Image Re-ranking with Long-Context Sequence Modeling

arXiv: https://arxiv.org/abs/2503.21772 · HTML: https://mlsanity.com/paper/2503.21772/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 80 references that are in the map.
- First posted on arXiv 27 Mar 2025.

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Ring Attention with Blockwise Transformers for Near-Infinite Context](https://mlsanity.com/paper/2310.01889/) (2023)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention](https://mlsanity.com/paper/2404.07143/) (2024)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [Span-based Joint Entity and Relation Extraction with Transformer Pre-training](https://mlsanity.com/paper/1909.07755/) (2019)
- [ProxyNCA++: Revisiting and Revitalizing Proxy Neighborhood Component Analysis](https://mlsanity.com/paper/2004.01113/) (2020)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures](https://mlsanity.com/paper/2403.02308/) (2024)
- [Semi-supervised sequence tagging with bidirectional language models](https://mlsanity.com/paper/1705.00108/) (2017)
- [Instance-level Image Retrieval using Reranking Transformers](https://mlsanity.com/paper/2103.12236/) (2021)
- [DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features](https://mlsanity.com/paper/2108.02927/) (2021)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [Visual Instance Retrieval with Deep Convolutional Networks](https://mlsanity.com/paper/1412.6574/) (2014)
- [Correlation Verification for Image Retrieval](https://mlsanity.com/paper/2204.01458/) (2022)
- [Global Features are All You Need for Image Retrieval and Reranking](https://mlsanity.com/paper/2308.06954/) (2023)

## Built on by (most cited first, 5 of 5)

- [Patch-wise Retrieval: A Bag of Practical Techniques for Instance-level Matching](https://mlsanity.com/paper/2512.12610/) (2025)
- [ELViS: Efficient Visual Similarity from Local Descriptors that Generalizes Across Domains](https://mlsanity.com/paper/2603.28603/) (2026)
- [Indexing Multimodal Language Models for Large-scale Image Retrieval](https://mlsanity.com/paper/2604.13268/) (2026)
- [Aerial-ground LiDAR place recognition with patch-level self-supervised learning and expanded reciprocal re-ranking](https://mlsanity.com/paper/2606.18583/) (2026)
- [Aggregating Neighbor Embedding Projection and Rank-Based Manifold Learning for Image Retrieval](https://mlsanity.com/paper/2609.01963/) (2026)

## Abstract (arXiv)

We introduce LOCORE, Long-Context Re-ranker, a model that takes as input local descriptors corresponding to an image query and a list of gallery images and outputs similarity scores between the query and each gallery image. This model is used for image retrieval, where typically a first ranking is performed with an efficient similarity measure, and then a shortlist of top-ranked images is re-ranked based on a more fine-grained similarity measure. Compared to existing methods that perform pair-wise similarity estimation with local descriptors or list-wise re-ranking with global descriptors, LOCORE is the first method to perform list-wise re-ranking with local descriptors. To achieve this, we leverage efficient long-context sequence models to effectively capture the dependencies between query and gallery images at the local-descriptor level. During testing, we process long shortlists with a sliding window strategy that is tailored to overcome the context size limitations of sequence models. Our approach achieves superior performance compared with other re-rankers on established image retrieval benchmarks of landmarks (ROxf and RPar), products (SOP), fashion items (In-Shop), and bird species (CUB-200) while having comparable latency to the pair-wise local descriptor re-rankers.