mlsanityData to 7 Oct 2026

Zero-Shot Composed Image Retrieval with Textual Inversion

arXiv 2303.15247 ↗First posted 27 Mar 2023cs.CV cs.CL cs.IRcited by 171

Not on the map. Why?

Cited by 31 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Composed Image Retrieval (CIR) aims to retrieve a target image based on a query composed of a reference image and a relative caption that describes the difference between the two images. The high effort and cost required for labeling datasets for CIR hamper the widespread usage of existing methods, as they rely on supervised learning. In this work, we propose a new task, Zero-Shot CIR (ZS-CIR), that aims to address CIR without requiring a labeled training dataset. Our approach, named zero-Shot composEd imAge Retrieval with textuaL invErsion (SEARLE), maps the visual features of the reference image into a pseudo-word token in CLIP token embedding space and integrates it with the relative caption. To support research on ZS-CIR, we introduce an open-domain benchmarking dataset named Composed Image Retrieval on Common Objects in context (CIRCO), which is the first dataset for CIR containing multiple ground truths for each query. The experiments show that SEARLE exhibits better performance than the baselines on the two main datasets for CIR tasks, FashionIQ and CIRR, and on the proposed CIRCO. The dataset, the code and the model are publicly available at https://github.com/miccunifi/SEARLE.

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

31 of 44 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2015VQA: Visual Question Answeringcited by 2,489
19 more
  1. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  2. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  3. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  4. 2017Mixed Precision Trainingcited by 1,135
  5. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  6. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  7. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  8. 2019Meshed-Memory Transformer for Image Captioningcited by 343
  9. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  10. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
  11. 2021Scaling Up Vision-Language Pre-training for Image Captioningcited by 196
  12. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
  13. 2017Automatic Spatially-aware Fashion Concept Discoverycited by 132
  14. 2022"This is my unicorn, Fluffy": Personalizing frozen vision-language representationscited by 95
  15. 2018Dialog-based Interactive Image Retrievalcited by 84
  16. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
  17. 2019Neural Naturalist: Generating Fine-Grained Image Comparisonscited by 36
  18. 2022Multiresolution Textual Inversioncited by 33
  19. 2021RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Networkcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 40; 2025: 50; 2026: 65.

171 papers.

Sort:

By area:

  1. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  2. 2023LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-Oncited by 151
  3. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  4. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  5. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 102
  6. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  7. 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
  8. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  9. 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
  10. 2023CoVR-2: Automatic Data Construction for Composed Video Retrievalcited by 52
  11. 2023Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrievalcited by 52
  12. 2024Spherical Linear Interpolation and Text-Anchoring for Zero-shot Composed Image Retrievalcited by 44
38 more
  1. 2025Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversioncited by 42
  2. 2025Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generationcited by 42
  3. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  4. 2023Zero-shot Composed Text-Image Retrievalcited by 31
  5. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  6. 2023Language-only Efficient Training of Zero-shot Composed Image Retrievalcited by 27
  7. 2025IDEA: Inverted Text with Cooperative Deformable Aggregation for Multi-modal Object Re-Identificationcited by 27
  8. 2024iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrievalcited by 24
  9. 2023Training-free Zero-shot Composed Image Retrieval with Local Concept Rerankingcited by 23
  10. 2024EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrievalcited by 21
  11. 2025CoLLM: A Large Language Model for Composed Image Retrievalcited by 21
  12. 2024Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negativescited by 20
  13. 2023Mapping Memes to Words for Multimodal Hateful Meme Classificationcited by 19
  14. 2023FreestyleRet: Retrieving Images from Style-Diversified Queriescited by 19
  15. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillationcited by 18
  16. 2024Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxycited by 18
  17. 2025Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrievalcited by 18
  18. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  19. 2025Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrievalcited by 17
  20. 2023Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrievalcited by 14
  21. 2024Image2Sentence based Asymmetrical Zero-shot Composed Image Retrievalcited by 14
  22. 2024Composed Image Retrieval for Remote Sensingcited by 14
  23. 2025SmartCLIP: Modular Vision-language Alignment with Identification Guaranteescited by 14
  24. 2024Visual Delta Generator with Large Multi-modal Models for Semi-supervised Composed Image Retrievalcited by 13
  25. 2024You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image Retrievalcited by 12
  26. 2024Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similaritycited by 12
  27. 2025ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoningcited by 12
  28. 2025Instance-Level Composed Image Retrievalcited by 12
  29. 2023Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Servicecited by 11
  30. 2024Leveraging Large Language Models for Multimodal Searchcited by 11
  31. 2025U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMscited by 10
  32. 2023OpenFashionCLIP: Vision-and-Language Contrastive Learning with Open-Source Fashion Datacited by 9
  33. 2024Text-Driven Traffic Anomaly Detection with Temporal High-Frequency Modeling in Driving Videoscited by 9
  34. 2024Personalized Representation from Personalized Generationcited by 9
  35. 2025ELIP: Enhanced Visual-Language Foundation Models for Image Retrievalcited by 9
  36. 2024HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labelscited by 8
  37. 2024Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignmentcited by 8
  38. 2025Personalization Toolkit: Training Free Personalization of Large Vision Language Modelscited by 8

50 most cited of 171 All 171 (JSON)