mlsanityData to 7 Oct 2026

Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval

arXiv 2302.03084 ↗First posted 6 Feb 2023cs.CVcited by 155

Not on the map. Why?

Cited by 29 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In Composed Image Retrieval (CIR), a user combines a query image with text to describe their intended target. Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image. Labeling such triplets is expensive and hinders broad applicability of CIR. In this work, we propose to study an important task, Zero-Shot Composed Image Retrieval (ZS-CIR), whose goal is to build a CIR model without requiring labeled triplets for training. To this end, we propose a novel method, called Pic2Word, that requires only weakly labeled image-caption pairs and unlabeled image datasets to train. Unlike existing supervised CIR models, our model trained on weakly labeled or unlabeled datasets shows strong generalization across diverse ZS-CIR tasks, e.g., attribute editing, object composition, and domain conversion. Our approach outperforms several supervised CIR methods on the common CIR benchmark, CIRR and Fashion-IQ. Code will be made publicly available at https://github.com/google-research/composed_image_retrieval.

Key facts. Cited by 155 papers in the map (data through 7 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

30 of 43 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  11. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
18 more
  1. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  2. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  3. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  4. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  5. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  6. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  7. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  8. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  9. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
  10. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
  11. 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
  12. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  13. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
  14. 2017Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrievalcited by 68
  15. 2020Modality-Agnostic Attention Fusion for visual search with text feedbackcited by 51
  16. 2022VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMixcited by 45
  17. 2022FashionViL: Fashion-Focused Vision-and-Language Representation Learningcited by 32
  18. 2022Probabilistic Compositional Embeddings for Multimodal Image Retrievalcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 19; 2024: 42; 2025: 48; 2026: 46.

155 papers.

Sort:

By area:

  1. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
  2. 2024E5-V: Universal Embeddings with Multimodal Large Language Modelscited by 123
  3. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  4. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  5. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 102
  6. 2023CompoDiff: Versatile Composed Image Retrieval With Latent Diffusioncited by 84
  7. 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
  8. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  9. 2023COLA: A Benchmark for Compositional Text-to-image Retrievalcited by 53
  10. 2023CoVR-2: Automatic Data Construction for Composed Video Retrievalcited by 52
  11. 2023Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrievalcited by 52
  12. 2024Spherical Linear Interpolation and Text-Anchoring for Zero-shot Composed Image Retrievalcited by 44
38 more
  1. 2023Bi-directional Training for Composed Image Retrieval via Text Prompt Learningcited by 43
  2. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  3. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  4. 2023Zero-shot Composed Text-Image Retrievalcited by 31
  5. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  6. 2024It's All About Your Sketch: Democratising Sketch Control in Diffusion Modelscited by 28
  7. 2023Language-only Efficient Training of Zero-shot Composed Image Retrievalcited by 27
  8. 2024Composed Video Retrieval via Enriched Context and Discriminative Embeddingscited by 26
  9. 2025Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrievalcited by 26
  10. 2024iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrievalcited by 24
  11. 2023Training-free Zero-shot Composed Image Retrieval with Local Concept Rerankingcited by 23
  12. 2024EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrievalcited by 21
  13. 2025CoLLM: A Large Language Model for Composed Image Retrievalcited by 21
  14. 2024Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negativescited by 20
  15. 2023Mapping Memes to Words for Multimodal Hateful Meme Classificationcited by 19
  16. 2024Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxycited by 18
  17. 2025Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrievalcited by 18
  18. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  19. 2025Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrievalcited by 17
  20. 2023From Text to Mask: Localizing Entities Using the Attention of Text-to-Image Diffusion Modelscited by 16
  21. 2023Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrievalcited by 14
  22. 2024Image2Sentence based Asymmetrical Zero-shot Composed Image Retrievalcited by 14
  23. 2024Composed Image Retrieval for Remote Sensingcited by 14
  24. 2024Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrievalcited by 13
  25. 2024BRIDGE: Bridging Gaps in Image Captioning Evaluation with Stronger Visual Cuescited by 13
  26. 2024Post-hoc Probabilistic Vision-Language Modelscited by 13
  27. 2024Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrievalcited by 13
  28. 2024You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrievalcited by 12
  29. 2024Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similaritycited by 12
  30. 2025Instance-Level Composed Image Retrievalcited by 12
  31. 2023Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Servicecited by 11
  32. 2024Leveraging Large Language Models for Multimodal Searchcited by 11
  33. 2024RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representationscited by 11
  34. 2024Personalized Representation from Personalized Generationcited by 9
  35. 2024HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labelscited by 8
  36. 2024Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignmentcited by 8
  37. 2025IDMR: Towards Instance-Driven Precise Visual Correspondence in Multimodal Retrievalcited by 8
  38. 2025QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrievalcited by 8

50 most cited of 155 All 155 (JSON)