# Compositional Learning of Image-Text Query for Image Retrieval

arXiv: https://arxiv.org/abs/2006.11149 · HTML: https://mlsanity.com/paper/2006.11149/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2020.

## Built on by (most cited first, 30 of 57)

- [Vision-by-Language for Training-Free Compositional Image Retrieval](https://mlsanity.com/paper/2310.09291/) (2023)
- ["This is my unicorn, Fluffy": Personalizing frozen vision-language representations](https://mlsanity.com/paper/2204.01694/) (2022)
- [CompoDiff: Versatile Composed Image Retrieval With Latent Diffusion](https://mlsanity.com/paper/2303.11916/) (2023)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features](https://mlsanity.com/paper/2308.11485/) (2023)
- [GeneCIS: A Benchmark for General Conditional Image Similarity](https://mlsanity.com/paper/2306.07969/) (2023)
- [Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization](https://mlsanity.com/paper/2211.07394/) (2022)
- [Bi-directional Training for Composed Image Retrieval via Text Prompt Learning](https://mlsanity.com/paper/2303.16604/) (2023)
- [Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encoder](https://mlsanity.com/paper/2305.16304/) (2023)
- [Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback](https://mlsanity.com/paper/2106.04128/) (2021)
- [RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network](https://mlsanity.com/paper/2104.03015/) (2021)
- [Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning](https://mlsanity.com/paper/2208.12526/) (2022)
- [Embedding Arithmetic of Multimodal Queries for Image Retrieval](https://mlsanity.com/paper/2112.03162/) (2021)
- [EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval](https://mlsanity.com/paper/2407.16658/) (2024)
- [Fine-grained Textual Inversion Network for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2503.19296/) (2025)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [On Leveraging Variational Graph Embeddings for Open World Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.11848/) (2022)
- [VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering](https://mlsanity.com/paper/2312.12273/) (2023)
- [NEUCORE: Neural Concept Reasoning for Composed Image Retrieval](https://mlsanity.com/paper/2310.01358/) (2023)
- [Image Search with Text Feedback by Additive Attention Compositional Learning](https://mlsanity.com/paper/2203.03809/) (2022)
- [TRACE: Task-Adaptive Reasoning and Representation Learning for Universal Multimodal Retrieval](https://mlsanity.com/paper/2603.02929/) (2026)
- [Language Guided Local Infiltration for Interactive Image Retrieval](https://mlsanity.com/paper/2304.07747/) (2023)
- [Dual Relation Alignment for Composed Image Retrieval](https://mlsanity.com/paper/2309.02169/) (2023)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [BOSS: Bottom-up Cross-modal Semantic Composition with Hybrid Counterfactual Training for Robust Content-based Image Retrieval](https://mlsanity.com/paper/2207.04211/) (2022)
- [Pseudo-triplet Guided Few-shot Composed Image Retrieval](https://mlsanity.com/paper/2407.06001/) (2024)
- [TMCIR: Token Merge Benefits Composed Image Retrieval](https://mlsanity.com/paper/2504.10995/) (2025)
- [Seeing the Abstract: Translating the Abstract Language for Vision Language Models](https://mlsanity.com/paper/2505.03242/) (2025)
- [DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrieval](https://mlsanity.com/paper/2505.17796/) (2025)

## Abstract (arXiv)

In this paper, we investigate the problem of retrieving images from a database based on a multi-modal (image-text) query. Specifically, the query text prompts some modification in the query image and the task is to retrieve images with the desired modifications. For instance, a user of an E-Commerce platform is interested in buying a dress, which should look similar to her friend's dress, but the dress should be of white color with a ribbon sash. In this case, we would like the algorithm to retrieve some dresses with desired modifications in the query dress. We propose an autoencoder based model, ComposeAE, to learn the composition of image and text query for retrieving images. We adopt a deep metric learning approach and learn a metric that pushes composition of source image and text query closer to the target images. We also propose a rotational symmetry constraint on the optimization problem. Our approach is able to outperform the state-of-the-art method TIRG citeTIRG on three benchmark datasets, namely: MIT-States, Fashion200k and Fashion IQ. In order to ensure fair comparison, we introduce strong baselines by enhancing TIRG method. To ensure reproducibility of the results, we publish our code here: urlhttps://github.com/ecom-research/ComposeAE.