mlsanityData to 14 Oct 2026

ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoning

arXiv 2503.10166 ↗First posted 13 Mar 2025cs.IR cs.AI cs.MMcited by 12

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

With the proliferation of images in online content, language-guided image retrieval (LGIR) has emerged as a research hotspot over the past decade, encompassing a variety of subtasks with diverse input forms. While the development of large multimodal models (LMMs) has significantly facilitated these tasks, existing approaches often address them in isolation, requiring the construction of separate systems for each task. This not only increases system complexity and maintenance costs, but also exacerbates challenges stemming from language ambiguity and complex image content, making it difficult for retrieval systems to provide accurate and reliable results. To this end, we propose ImageScope, a training-free, three-stage framework that leverages collective reasoning to unify LGIR tasks. The key insight behind the unification lies in the compositional nature of language, which transforms diverse LGIR tasks into a generalized text-to-image retrieval process, along with the reasoning of LMMs serving as a universal verification to refine the results. To be specific, in the first stage, we improve the robustness of the framework by synthesizing search intents across varying levels of semantic granularity using chain-of-thought (CoT) reasoning. In the second and third stages, we then reflect on retrieval results by verifying predicate propositions locally, and performing pairwise evaluations globally. Experiments conducted on six LGIR datasets demonstrate that ImageScope outperforms competitive baselines. Comprehensive evaluations and ablation studies further confirm the effectiveness of our design.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 69 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

31 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023MAKE: Vision-Language Pre-training based Product Retrieval in Taobao Search30 Jan

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  9. 2015VQA: Visual Question Answeringcited by 2,506
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  11. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  12. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
18 more
  1. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  3. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  4. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  5. 2016Visual Dialogcited by 377
  6. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
  7. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  8. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 176
  9. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 103
  10. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 71
  11. 2022FashionViL: Fashion-Focused Vision-and-Language Representation Learningcited by 32
  12. 2024Improving Retrieval Augmented Language Model with Self-Reasoningcited by 28
  13. 2024Interactive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approachcited by 25
  14. 2019Improving Generative Visual Dialog by Answering Diverse Questionscited by 21
  15. 2024HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labelscited by 8
  16. 2023Language Guided Local Infiltration for Interactive Image Retrievalcited by 6
  17. 2023Unified Vision-Language Representation Modeling for E-Commerce Same-Style Products Retrievalcited by 5
  18. 2017Why People Search for Images using Web Search Enginescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 10.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Indexing Multimodal Language Models for Large-scale Image Retrieval
  2. 2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models
  3. 2025CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning
  4. 2024TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Models
  5. 2026CoCo-IR: Contextual Composed Image Retrieval
  6. 2024Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark
  7. 2026Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval
  8. 2024Document Haystacks: Vision-Language Reasoning Over Piles of 1000+ Documents