mlsanityData to 7 Oct 2026

Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrieval

arXiv 2403.16005 ↗First posted 24 Mar 2024cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

We study the zero-shot Composed Image Retrieval (ZS-CIR) task, which is to retrieve the target image given a reference image and a description without training on the triplet datasets. Previous works generate pseudo-word tokens by projecting the reference image features to the text embedding space. However, they focus on the global visual representation, ignoring the representation of detailed attributes, e.g., color, object number and layout. To address this challenge, we propose a Knowledge-Enhanced Dual-stream zero-shot composed image retrieval framework (KEDs). KEDs implicitly models the attributes of the reference images by incorporating a database. The database enriches the pseudo-word tokens by providing relevant images and captions, emphasizing shared attribute information in various aspects. In this way, KEDs recognizes the reference image from diverse perspectives. Moreover, KEDs adopts an extra stream that aligns pseudo-word tokens with textual concepts, leveraging pseudo-triplets mined from image-text pairs. The pseudo-word tokens generated in this stream are explicitly aligned with fine-grained semantics in the text embedding space. Extensive experiments on widely used benchmarks, i.e. ImageNet-R, COCO object, Fashion-IQ and CIRR, show that KEDs outperforms previous zero-shot composed image retrieval methods.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 59 of its 81 references that are in the map. First posted on arXiv 24 Mar 2024.

Builds on

59 of 81 references.

References · 59

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2017Billion-scale similarity search with GPUscited by 2,477
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  12. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
38 more
  1. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  2. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  3. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  4. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  5. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  6. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classificationcited by 355
  7. 2020VirTex: Learning Visual Representations from Textual Annotationscited by 338
  8. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  9. 2023WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentationcited by 311
  10. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 296
  11. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  12. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
  13. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  14. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  15. 2023Key-Locked Rank One Editing for Text-to-Image Personalizationcited by 194
  16. 2022CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labelscited by 177
  17. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
  18. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
  19. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  20. 2016Deep Image Retrieval: Learning global representations for image searchcited by 147
  21. 2015Natural Language Object Retrievalcited by 124
  22. 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
  23. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  24. 2022CLIP-Event: Connecting Text and Images with Event Structurescited by 86
  25. 2023DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Trainingcited by 84
  26. 2023CompoDiff: Versatile Composed Image Retrieval With Latent Diffusioncited by 84
  27. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
  28. 2022K-LITE: Learning Transferable Visual Models with External Knowledgecited by 79
  29. 2023Vista-LLaMA: Reducing Hallucination in Video Language Models via Equal Distance to Visual Tokenscited by 70
  30. 2022CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classificationcited by 65
  31. 2022FiD-Light: Efficient and Effective Retrieval-Augmented Text Generationcited by 63
  32. 2024Volumetric Environment Representation for Vision-Language Navigationcited by 63
  33. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  34. 2019Auto-ReID: Searching for a Part-aware ConvNet for Person Re-Identificationcited by 51
  35. 2020Modality-Agnostic Attention Fusion for visual search with text feedbackcited by 51
  36. 2022ASIF: Coupled Data Turns Unimodal Models to Multimodal Without Trainingcited by 45
  37. 2023EdaDet: Open-Vocabulary Object Detection Using Early Dense Alignmentcited by 44
  38. 2017Self-supervised learning of visual features through embedding images into text topic spacescited by 43

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 13; 2026: 13.

36 papers.

Sort:

By area:

  1. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  2. 2024Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activitycited by 22
  3. 2024Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxycited by 18
  4. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  5. 2025Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrievalcited by 17
  6. 2025HarmonySet: A Comprehensive Dataset for Understanding Video-Music Semantic Alignment and Temporal Synchronizationcited by 14
  7. 2024InfiniDreamer: Arbitrarily Long Human Motion Generation via Segment Score Distillationcited by 12
  8. 2024HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labelscited by 8
  9. 2025Multimodal Reasoning Agent for Zero-Shot Composed Image Retrievalcited by 7
  10. 2024Localizing Events in Videos with Multimodal Queriescited by 6
  11. 2024A Survey of Multimodal Composite Editing and Retrievalcited by 5
  12. 2025MLLM-Guided VLM Fine-Tuning with Joint Inference for Zero-Shot Composed Image Retrievalcited by 5
24 more
  1. 2024Composed Image Retrieval for Training-Free Domain Conversioncited by 4
  2. 2025DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrievalcited by 4
  3. 2025PDV: Prompt Directional Vectors for Zero-shot Composed Image Retrievalcited by 3
  4. 2024An Efficient Post-hoc Framework for Reducing Task Discrepancy of Text Encoders for Composed Image Retrievalcited by 2
  5. 2024Modality and Task Adaptation for Enhanced Zero-shot Composed Image Retrievalcited by 2
  6. 2025Composed Multi-modal Retrieval: A Survey of Approaches and Applicationscited by 2
  7. 2025From Mapping to Composing: A Two-Stage Framework for Zero-shot Composed Image Retrievalcited by 2
  8. 2025Automatic Synthesis of High-Quality Triplet Data for Composed Image Retrievalcited by 2
  9. 2026Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learningcited by 2
  10. 2026V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrievalcited by 2
  11. 2026UniCVR: From Alignment to Reranking for Unified Zero-Shot Composed Visual Retrievalcited by 2
  12. 2025Data-Efficient Generalization for Zero-shot Composed Image Retrievalcited by 1
  13. 2025SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking for Training-free Zero-Shot Composed Image Retrievalcited by 1
  14. 2026G-MIXER: Geodesic Mixup-based Implicit Semantic Expansion and Explicit Semantic Re-ranking for Zero-Shot Composed Image Retrievalcited by 1
  15. 2025Learning Compatible Multi-Prize Subnetworks for Asymmetric Retrievalcited by 0
  16. 2026Mask-Guided Multi-Task Network for Face Attribute Recognitioncited by 0
  17. 2026Fine-Grained Zero-Shot Composed Image Retrieval with Complementary Visual-Semantic Integrationcited by 0
  18. 2026DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrievalcited by 0
  19. 2026RAAP: Retrieval-Augmented Affordance Prediction with Cross-Image Action Alignmentcited by 0
  20. 2026A Sanity Check on Composed Image Retrievalcited by 0
  21. 2026AlbumFill: Album-Guided Reasoning and Retrieval for Personalized Image Completioncited by 0
  22. 2026Decoupling Endpoint and Semantic Transition Learning for Zero-Shot Composed Image Retrievalcited by 0
  23. 2026Thinking Before Retrieving: Robust Zero-Shot Composed Image Retrieval via Strategic Planning and Self-Criticismcited by 0
  24. 2026MulVec: Fine-Grained Role-Aware Matching for Training-Free Zero-Shot Composed Image Retrievalcited by 0