mlsanityData to 14 Oct 2026

Training-Free Pseudo-Fusion for Composed Image Retrieval with Diffusion Models and Multimodal Large Language Models

arXiv 2608.23102 ↗First posted 24 Aug 2026cs.CV cs.IRcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more
Abstract (arXiv)

Composed Image Retrieval (CIR) is an emerging paradigm in content-based image retrieval that enables users to formulate compositional queries by combining a reference image with an auxiliary modality, usually text-based. This approach supports fine-grained search where the target image shares structural elements with the user-provided image while incorporating the modifications specified by the auxiliary text. Conventional CIR methods rely on multimodal fusion to combine visual and textual features into a joint query embedding, which requires training modules that align composed queries with the targets. In this work, we propose PeFuse (for pseudo-fusion), a training-free framework that leverages pretrained Diffusion Models and Multimodal Large Language Models to bridge modalities via generative conversion. We introduce two novel strategies: uni-directional and bi-directional conversion, which convert CIR into four single-modality retrieval problems. These methods reformulate CIR as either intra-modal or cross-modal single-query retrieval tasks, bypassing the need for dedicated task-specific training. Extensive experiments on standard benchmarks demonstrate that converting CIR into text-to-image retrieval tasks is more effective than alternative conversion strategies, achieving competitive or superior performance compared with state-of-the-art methods, while maintaining high flexibility thanks to replaceable components of the conversion pipeline. These results highlight the effectiveness of the pseudo-fusion paradigm for zero-shot CIR. Our code is publicly available at: https://github.com/StevenXuf/PeFuse4CIR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 50 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

43 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoning13 Mar

Other references · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2024The Llama 3 Herd of Modelscited by 14,715
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  9. 2020Denoising Diffusion Implicit Modelscited by 9,781
  10. 2023Visual Instruction Tuningcited by 7,869
  11. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
30 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  6. 2017Billion-scale similarity search with GPUscited by 2,497
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  8. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  9. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  10. 2023Adversarial Diffusion Distillationcited by 759
  11. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
  12. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  13. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 176
  14. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 156
  15. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 103
  16. 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
  17. 2023GeneCIS: A Benchmark for General Conditional Image Similaritycited by 47
  18. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 43
  19. 2023Language-only Efficient Training of Zero-shot Composed Image Retrievalcited by 27
  20. 2024Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrievalcited by 24
  21. 2025CoLLM: A Large Language Model for Composed Image Retrievalcited by 21
  22. 2024Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negativescited by 20
  23. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  24. 2024Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similaritycited by 12
  25. 2024HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labelscited by 8
  26. 2025Multimodal Reasoning Agent for Zero-Shot Composed Image Retrievalcited by 8
  27. 2025ConText-CIR: Learning from Concepts in Text for Composed Image Retrievalcited by 7
  28. 2025Multimodal Representation Alignment for Cross-modal Information Retrievalcited by 2
  29. 2025Automatic Synthesis of High-Quality Triplet Data for Composed Image Retrievalcited by 2
  30. 2024EUFCC-CIR: a Composed Image Retrieval Dataset for GLAM Collectionscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Sanity Check on Composed Image Retrieval
  2. 2026Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval
  3. 2026ADaFuSE: Adaptive Diffusion-generated Image and Text Fusion for Interactive Text-to-Image Retrieval
  4. 2025Soft Filtering: Guiding Zero-shot Composed Image Retrieval with Prescriptive and Proscriptive Constraints
  5. 2025Instance-Level Composed Image Retrieval
  6. 2024Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrieval
  7. 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval
  8. 2024Compositional Image Retrieval via Instruction-Aware Contrastive Learning