mlsanityData to 14 Oct 2026

QuARI: Query Adaptive Retrieval Improvement

arXiv 2505.21647 ↗First posted 27 May 2025cs.CV cs.LGcited by 2

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Massive-scale pretraining has made vision-language models increasingly popular for image-to-image and text-to-image retrieval across a broad collection of domains. However, these models do not perform well when used for challenging retrieval tasks, such as instance retrieval in very large-scale image collections. Recent work has shown that linear transformations of VLM features trained for instance retrieval can improve performance by emphasizing subspaces that relate to the domain of interest. In this paper, we explore a more extreme version of this specialization by learning to map a given query to a query-specific feature space transformation. Because this transformation is linear, it can be applied with minimal computational cost to millions of image embeddings, making it effective for large-scale retrieval or re-ranking. Results show that this method consistently outperforms state-of-the-art alternatives, including those that require many orders of magnitude more computation at query time.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 50 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

36 of 50 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025ILIAS: Instance-Level Image retrieval At Scale17 Feb
  2. 2024INQUIRE: A Natural World Text-to-Image Retrieval Benchmark4 Nov

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2016Layer Normalizationcited by 5,669
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  11. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  12. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
22 more
  1. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  2. 2016HyperNetworkscited by 1,027
  3. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  4. 2018Recognition in Terra Incognitacited by 658
  5. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  6. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  7. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 400
  8. 2014CNN Features off-the-shelf: an Astounding Baseline for Recognitioncited by 316
  9. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 225
  10. 2021Density-aware Chamfer Distance as a Comprehensive Metric for Point Cloud Completioncited by 112
  11. 2022Transformers as Meta-Learners for Implicit Neural Representationscited by 79
  12. 2021Instance-level Image Retrieval using Reranking Transformerscited by 63
  13. 2024Improving fine-grained understanding in image-text pre-trainingcited by 44
  14. 2023Global Features are All You Need for Image Retrieval and Rerankingcited by 41
  15. 2023Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representationscited by 28
  16. 2022Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Trainingcited by 24
  17. 2024BioTrove: A Large Curated Image Dataset Enabling AI for Biodiversitycited by 20
  18. 2024AMES: Asymmetric and Memory-Efficient Similarity Estimation for Instance-level Retrievalcited by 16
  19. 2019Hotels-50K: A Global Hotel Recognition Datasetcited by 13
  20. 2022APE: Aligning Pretrained Encoders to Quickly Learn Aligned Multimodal Representationscited by 9
  21. 2023FORB: A Flat Object Retrieval Benchmark for Universal Image Embeddingcited by 9
  22. 2023Region-centric Image-Language Pretraining for Open-Vocabulary Detectioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Geometry-Aware CLIP Retrieval via Local Cross-Modal Alignment and Steeringcited by 0
  2. 2026Trajectory-aware Cross-view Geo-localization with Sequential Observationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models
  2. 2025A Little More Like This: Text-to-Image Retrieval with Vision-Language Models Using Relevance Feedback
  3. 2026Indexing Multimodal Language Models for Large-scale Image Retrieval
  4. 2022HiVLP: Hierarchical Vision-Language Pre-Training for Fast Image-Text Retrieval
  5. 2024RoRA-VLM: Robust Retrieval-Augmented Vision Language Models
  6. 2025Enhancing Subsequent Video Retrieval via Vision-Language Models (VLMs)
  7. 2025Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders
  8. 2025SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking for Training-free Zero-Shot Composed Image Retrieval