ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
Abstract (arXiv)
Composed Image Retrieval (CIR) aims to retrieve target images based on a hybrid query comprising a reference image and a modification text. Early dual-tower Vision-Language Models (VLMs) struggle with cross-modality compositional reasoning required for this task. While adapting generative Multimodal Large Language Models (MLLMs) for retrieval offers a promising direction, we identify that this strategy overlooks a fundamental issue: compressing a generative MLLM into a single-embedding discriminative retriever triggers a paradigm conflict, which leads to Capability Degradation - the deterioration of native fine-grained reasoning after retrieval adaptation. To address this challenge, we propose ReCALL, a model-agnostic framework that follows a diagnose-generate-refine pipeline: First, we diagnose cognitive blind spots of the retriever via self-guided informative instance mining. Next, we generate corrective instructions and triplets by prompting the foundation MLLM and conduct quality control with VQA-based consistency filtering. Finally, we refine the retriever through continual training on these triplets with a grouped contrastive scheme, thereby internalizing fine-grained visual-semantic distinctions and realigning the discriminative embedding space of retriever with intrinsic compositional reasoning within the MLLM. Extensive experiments on CIRR and FashionIQ show that ReCALL consistently recalibrates degraded capabilities and achieves state-of-the-art performance. Code is available at https://github.com/RemRico/Recall.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 52 of its 77 references that are in the map. First posted on arXiv 2 Feb 2026.
Builds on
52 of 77 references.
Closest ancestors · 7
Same direction, newest first
- 2024Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval15 Dec
- 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistant2 Dec
- 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs4 Nov
- 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks7 Oct
- 2024TIGeR: Unifying Text-to-Image Generation and Retrieval with Large Multimodal Models9 Jun
- 2023FashionSAP: Symbols and Attributes Prompt for Fine-grained Fashion Vision-Language Pre-training11 Apr
- 2023FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks4 Mar
Other references · 45
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
- 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
33 more
- 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
- 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
- 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
- 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
- 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 238
- 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
- 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
- 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
- 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
- 2016Deep Image Retrieval: Learning global representations for image searchcited by 147
- 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
- 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
- 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
- 2020Compositional Learning of Image-Text Query for Image Retrievalcited by 57
- 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
- 2023CoVR-2: Automatic Data Construction for Composed Video Retrievalcited by 52
- 2023Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrievalcited by 52
- 2022Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularizationcited by 45
- 2023Data Roaming and Quality Assessment for Composed Image Retrievalcited by 45
- 2023Target-Guided Composed Image Retrievalcited by 36
- 2023Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encodercited by 35
- 2025Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineeringcited by 34
- 2023Self-Training Boosted Multi-Factor Matching Network for Composed Image Retrievalcited by 22
- 2024CaLa: Complementary Association Learning for Augmenting Composed Image Retrievalcited by 21
- 2025Improving Reasoning Performance in Large Language Models via Representation Engineeringcited by 21
- 2025VINCIE: Unlocking In-context Image Editing from Videocited by 17
- 2025CoTMR: Chain-of-Thought Multi-Scale Reasoning for Training-Free Zero-Shot Composed Image Retrievalcited by 15
- 2024Composing Object Relations and Attributes for Image-Text Matchingcited by 11
- 2024SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretrainingcited by 8
- 2025QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrievalcited by 8
- 2025ConText-CIR: Learning from Concepts in Text for Composed Image Retrievalcited by 7
- 2025Referring Expression Instance Retrieval and A Strong End-to-End Baselinecited by 6
- 2023The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Modelscited by 5
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning
- 2025CoLLM: A Large Language Model for Composed Image Retrieval
- 2026Indexing Multimodal Language Models for Large-scale Image Retrieval
- 2025Enhancing Supervised Composed Image Retrieval via Reasoning-Augmented Representation Engineering
- 2026Training-Free Pseudo-Fusion for Composed Image Retrieval with Diffusion Models and Multimodal Large Language Models
- 2024Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond
- 2024Compositional Image Retrieval via Instruction-Aware Contrastive Learning
- 2026MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval