mlsanityData to 7 Oct 2026

Composing Text and Image for Image Retrieval - An Empirical Odyssey

arXiv 1812.07119 ↗First posted 18 Dec 2018cs.CVcited by 234

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we study the task of image retrieval, where the input query is specified in the form of an image plus some text that describes desired modifications to the input image. For example, we may present an image of the Eiffel tower, and ask the system to find images which are visually similar but are modified in small ways, such as being taken at nighttime instead of during the day. To tackle this task, we learn a similarity metric between a target image and a source image plus source text, an embedding and composing function such that target image feature is close to the source image plus text composition feature. We propose a new way to combine image and text using such function that is designed for the retrieval task. We show this outperforms existing approaches on 3 different datasets, namely Fashion-200k, MIT-States and a new synthetic dataset we create based on CLEVR. We also show that our approach can be used to classify input queries, in addition to image retrieval.

Key facts. Cited by 234 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 26; 2023: 48; 2024: 39; 2025: 51; 2026: 47.

234 papers.

Sort:

By area:

  1. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  2. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
  3. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
  4. 2024MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructionscited by 119
  5. 2023Vision-by-Language for Training-Free Compositional Image Retrievalcited by 102
  6. 2021Learning to Predict Visual Attributes in the Wildcited by 95
  7. 2023CompoDiff: Versatile Composed Image Retrieval With Latent Diffusioncited by 84
  8. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 82
  9. 2024VISTA: Visualized Text Embedding For Universal Multi-Modal Retrievalcited by 67
  10. 2023Inst-Inpaint: Instructing to Remove Objects with Diffusion Modelscited by 66
  11. 2023Sentence-level Prompts Benefit Composed Image Retrievalcited by 59
  12. 2021Distilling Audio-Visual Knowledge by Compositional Contrastive Learningcited by 53
38 more
  1. 2023Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Featurescited by 52
  2. 2023CoVR-2: Automatic Data Construction for Composed Video Retrievalcited by 52
  3. 2023Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrievalcited by 52
  4. 2023GeneCIS: A Benchmark for General Conditional Image Similaritycited by 46
  5. 2022Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularizationcited by 45
  6. 2023Data Roaming and Quality Assessment for Composed Image Retrievalcited by 45
  7. 2024Spherical Linear Interpolation and Text-Anchoring for Zero-shot Composed Image Retrievalcited by 44
  8. 2023Bi-directional Training for Composed Image Retrieval via Text Prompt Learningcited by 43
  9. 2025FineCIR: Explicit Parsing of Fine-Grained Modification Semantics for Composed Image Retrievalcited by 41
  10. 2025OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrievalcited by 40
  11. 2023PoseFix: Correcting 3D Human Poses with Natural Languagecited by 38
  12. 2023FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Taskscited by 37
  13. 2024Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrievalcited by 36
  14. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  15. 2023Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encodercited by 35
  16. 2021ZFlow: Gated Appearance Flow-based Virtual Try-on with 3D Priorscited by 34
  17. 2021Discrete Representations Strengthen Vision Transformer Robustnesscited by 34
  18. 2022FashionViL: Fashion-Focused Vision-and-Language Representation Learningcited by 32
  19. 2025HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrievalcited by 32
  20. 2023Zero-shot Composed Text-Image Retrievalcited by 31
  21. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  22. 2021Conversational Fashion Image Retrieval via Multiturn Natural Language Feedbackcited by 29
  23. 2022Probabilistic Compositional Embeddings for Multimodal Image Retrievalcited by 29
  24. 2022A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketchcited by 28
  25. 2021RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Networkcited by 27
  26. 2023Language-only Efficient Training of Zero-shot Composed Image Retrievalcited by 27
  27. 2022Progressive Learning for Image Retrieval with Hybrid-Modality Queriescited by 26
  28. 2023Image Clustering Conditioned on Text Criteriacited by 26
  29. 2023Plan, Posture and Go: Towards Open-World Text-to-Motion Generationcited by 26
  30. 2024Composed Video Retrieval via Enriched Context and Discriminative Embeddingscited by 26
  31. 2026INTENT: Invariance and Discrimination-aware Noise Mitigation for Robust Composed Image Retrievalcited by 26
  32. 2023Chatting Makes Perfect: Chat-based Image Retrievalcited by 25
  33. 2024Interactive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approachcited by 25
  34. 2021Extending CLIP for Category-to-image Retrieval in E-commercecited by 24
  35. 2024iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrievalcited by 24
  36. 2023Training-free Zero-shot Composed Image Retrieval with Local Concept Rerankingcited by 23
  37. 2021Embedding Arithmetic of Multimodal Queries for Image Retrievalcited by 21
  38. 2024EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrievalcited by 21

50 most cited of 234 All 234 (JSON)