mlsanityData to 14 Oct 2026

Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features

arXiv 2308.11485 ↗First posted 22 Aug 2023cs.CVcited by 52

Not on the map. Why?

Abstract (arXiv)

Given a query composed of a reference image and a relative caption, the Composed Image Retrieval goal is to retrieve images visually similar to the reference one that integrates the modifications expressed by the caption. Given that recent research has demonstrated the efficacy of large-scale vision and language pre-trained (VLP) models in various tasks, we rely on features from the OpenAI CLIP model to tackle the considered task. We initially perform a task-oriented fine-tuning of both CLIP encoders using the element-wise sum of visual and textual features. Then, in the second stage, we train a Combiner network that learns to combine the image-text features integrating the bimodal information and providing combined features used to perform the retrieval. We use contrastive learning in both stages of training. Starting from the bare CLIP features as a baseline, experimental results show that the task-oriented fine-tuning and the carefully crafted Combiner network are highly effective and outperform more complex state-of-the-art approaches on FashionIQ and CIRR, two popular and challenging datasets for composed image retrieval. Code and pre-trained models are available at https://github.com/ABaldrati/CLIP4Cir

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 22 Aug 2023.

Builds on

35 of 59 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  8. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  9. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  10. 2017Mixed Precision Trainingcited by 1,156
  11. 2016Training Deep Nets with Sublinear Memory Costcited by 848
  12. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
23 more
  1. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 459
  2. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 251
  3. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 236
  4. 2021CLIP2Video: Mastering Video-Text Retrieval via Image CLIPcited by 225
  5. 2017Automatic Spatially-aware Fashion Concept Discoverycited by 134
  6. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  7. 2022CLIP-Event: Connecting Text and Images with Event Structurescited by 86
  8. 2018Dialog-based Interactive Image Retrievalcited by 84
  9. 2022ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similaritycited by 83
  10. 2016SIFT Meets CNN: A Decade Survey of Instance Retrievalcited by 67
  11. 2022CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classificationcited by 65
  12. 2020Compositional Learning of Image-Text Query for Image Retrievalcited by 57
  13. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57
  14. 2020Modality-Agnostic Attention Fusion for visual search with text feedbackcited by 51
  15. 2021Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretrainingcited by 42
  16. 2022CLIP-TD: CLIP Targeted Distillation for Vision-Language Taskscited by 32
  17. 2022FashionViL: Fashion-Focused Vision-and-Language Representation Learningcited by 32
  18. 2021Conversational Fashion Image Retrieval via Multiturn Natural Language Feedbackcited by 29
  19. 2021RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Networkcited by 27
  20. 2020SAC: Semantic Attention Composition for Text-Conditioned Image Retrievalcited by 23
  21. 2021Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillationcited by 22
  22. 2020CurlingNet: Compositional Learning between Images and Text for Fashion IQ Datacited by 21
  23. 2015WhittleSearch: Interactive Image Search with Relative Attribute Feedbackcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 9; 2025: 23; 2026: 19.

52 papers.

Sort:

By area:

  1. 2024TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negativescited by 58
  2. 2024iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrievalcited by 24
  3. 2024GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Promptingcited by 22
  4. 2024Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negativescited by 20
  5. 2024UniFashion: A Unified Vision-Language Model for Multimodal Fashion Retrieval and Generationcited by 17
  6. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  7. 2026TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrievalcited by 15
  8. 2024Image2Sentence based Asymmetrical Zero-shot Composed Image Retrievalcited by 14
  9. 2023VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answeringcited by 11
  10. 2025VADB: A Large-Scale Video Aesthetic Database with Professional and Multi-Dimensional Annotationscited by 9
  11. 2025Multimodal Reasoning Agent for Zero-Shot Composed Image Retrievalcited by 8
  12. 2025Scale Up Composed Image Retrieval Learning via Modification Text Generationcited by 7
38 more
  1. 2025ConText-CIR: Learning from Concepts in Text for Composed Image Retrievalcited by 7
  2. 2024AIGC for Industrial Time Series: From Deep Generative Models to Large Generative Modelscited by 6
  3. 2024A Survey of Multimodal Composite Editing and Retrievalcited by 5
  4. 2025GENIUS: A Generative Framework for Universal Multimodal Searchcited by 5
  5. 2025MLLM-Guided VLM Fine-Tuning with Joint Inference for Zero-Shot Composed Image Retrievalcited by 5
  6. 2025ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrievalcited by 5
  7. 2025From Generator to Embedder: Harnessing Innate Abilities of Multimodal LLMs via Building Zero-Shot Discriminative Embedding Modelcited by 5
  8. 2025good4cir: Generating Detailed Synthetic Captions for Composed Image Retrievalcited by 4
  9. 2025DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrievalcited by 4
  10. 2025MoRA: Missing Modality Low-Rank Adaptation for Visual Recognitioncited by 4
  11. 2025CultureCLIP: Empowering CLIP with Cultural Awareness through Synthetic Images and Contextualized Captionscited by 3
  12. 2025CalibCLIP: Contextual Calibration of Dominant Semantics for Text-Driven Image Retrievalcited by 3
  13. 2026ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrievalcited by 3
  14. 2025Composed Multi-modal Retrieval: A Survey of Approaches and Applicationscited by 2
  15. 2025Does CLIP perceive art the same way we do?cited by 2
  16. 2025OneVision: An End-to-End Generative Framework for Multi-view E-commerce Vision Searchcited by 2
  17. 2026SGMatch: Semantic-Guided Non-Rigid Shape Matching with Flow Regularizationcited by 2
  18. 2025GRAPE: Let GRPO Supervise Query Rewriting by Ranking for Retrievalcited by 1
  19. 2025SQUARE: Semantic Query-Augmented Fusion and Efficient Batch Reranking for Training-free Zero-Shot Composed Image Retrievalcited by 1
  20. 2026OSCAR: Optimization-Steered Agentic Planning for Composed Image Retrievalcited by 1
  21. 2026Never Seen Before: Benchmarking Genuine Zero-Shot Composed Image Retrieval with Consistent Video-Sourced Datasetscited by 1
  22. 2026FiRE: Enhancing MLLMs with Fine-Grained Context Learning for Complex Image Retrievalcited by 1
  23. 2024Fusion Embedding for Pose-Guided Person Image Synthesis with Diffusion Modelcited by 0
  24. 2025Triplet Synthesis For Enhancing Composed Image Retrieval via Counterfactual Image Generationcited by 0
  25. 2025AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMscited by 0
  26. 2025HMID-Net: An Exploration of Masked Image Modeling and Knowledge Distillation in Hyperbolic Spacecited by 0
  27. 2025Composed Object Retrieval: Object-level Retrieval via Composed Expressionscited by 0
  28. 2026DQE-CIR: Distinctive Query Embeddings through Learnable Attribute Weights and Target Relative Negative Sampling in Composed Image Retrievalcited by 0
  29. 2026PinPoint: Evaluation of Composed Image Retrieval with Explicit Negatives, Multi-Image Queries, and Paraphrase Testingcited by 0
  30. 2026FBCIR: Balancing Cross-Modal Focuses in Composed Image Retrievalcited by 0
  31. 2026Beyond Semantic Search: Towards Referential Anchoring in Composed Image Retrievalcited by 0
  32. 2026WRF4CIR: Weight-Regularized Fine-Tuning Network for Composed Image Retrievalcited by 0
  33. 2026Adjustable Text-Guided Backdoor Attacks with Natural-Word Triggers on Multimodal Pretrained Modelscited by 0
  34. 2026A Sanity Check on Composed Image Retrievalcited by 0
  35. 2026Decoupling Endpoint and Semantic Transition Learning for Zero-Shot Composed Image Retrievalcited by 0
  36. 2026VietFashion: Benchmarking Sketch-Text Composed Image Retrieval for Cultural Outfitscited by 0
  37. 2026Exploring Multi-Modal Large Language Models and Two-Stage Fine-Tuning for Fashion Image Retrievalcited by 0
  38. 2026Diverse-Intent Multi-Turn Fashion Image Retrievalcited by 0

50 most cited of 52 All 52 (JSON)