Training-Free Pseudo-Fusion for Composed Image Retrieval with Diffusion Models and Multimodal Large Language Models
On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
also in 1 more
Abstract (arXiv)
Composed Image Retrieval (CIR) is an emerging paradigm in content-based image retrieval that enables users to formulate compositional queries by combining a reference image with an auxiliary modality, usually text-based. This approach supports fine-grained search where the target image shares structural elements with the user-provided image while incorporating the modifications specified by the auxiliary text. Conventional CIR methods rely on multimodal fusion to combine visual and textual features into a joint query embedding, which requires training modules that align composed queries with the targets. In this work, we propose PeFuse (for pseudo-fusion), a training-free framework that leverages pretrained Diffusion Models and Multimodal Large Language Models to bridge modalities via generative conversion. We introduce two novel strategies: uni-directional and bi-directional conversion, which convert CIR into four single-modality retrieval problems. These methods reformulate CIR as either intra-modal or cross-modal single-query retrieval tasks, bypassing the need for dedicated task-specific training. Extensive experiments on standard benchmarks demonstrate that converting CIR into text-to-image retrieval tasks is more effective than alternative conversion strategies, achieving competitive or superior performance compared with state-of-the-art methods, while maintaining high flexibility thanks to replaceable components of the conversion pipeline. These results highlight the effectiveness of the pseudo-fusion paradigm for zero-shot CIR. Our code is publicly available at: https://github.com/StevenXuf/PeFuse4CIR.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 50 references that are in the map. First posted on arXiv 24 Aug 2026.
Builds on
43 of 50 references.
Closest ancestors · 1
Same direction, newest first
- 2025ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoning13 Mar
Other references · 42
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2023Visual Instruction Tuningcited by 7,869
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
30 more
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
- 2023Adversarial Diffusion Distillationcited by 759
- 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
- 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
- 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 176
- 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 156
- 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 103
- 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
- 2023GeneCIS: A Benchmark for General Conditional Image Similaritycited by 47
- 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
- 2023Language-only Efficient Training of Zero-shot Composed Image Retrievalcited by 27
- 2024Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrievalcited by 24
- 2025CoLLM: A Large Language Model for Composed Image Retrievalcited by 21
- 2024Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negativescited by 20
- 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
- 2024Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similaritycited by 12
- 2024HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labelscited by 8
- 2025Multimodal Reasoning Agent for Zero-Shot Composed Image Retrievalcited by 8
- 2025ConText-CIR: Learning from Concepts in Text for Composed Image Retrievalcited by 7
- 2025Multimodal Representation Alignment for Cross-modal Information Retrievalcited by 2
- 2025Automatic Synthesis of High-Quality Triplet Data for Composed Image Retrievalcited by 2
- 2024EUFCC-CIR: a Composed Image Retrieval Dataset for GLAM Collectionscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Sanity Check on Composed Image Retrieval
- 2026Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval
- 2026ADaFuSE: Adaptive Diffusion-generated Image and Text Fusion for Interactive Text-to-Image Retrieval
- 2025Soft Filtering: Guiding Zero-shot Composed Image Retrieval with Prescriptive and Proscriptive Constraints
- 2025Instance-Level Composed Image Retrieval
- 2024Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval
- 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval
- 2024Compositional Image Retrieval via Instruction-Aware Contrastive Learning