# Bi-directional Training for Composed Image Retrieval via Text Prompt Learning

arXiv: https://arxiv.org/abs/2303.16604 · HTML: https://mlsanity.com/paper/2303.16604/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 29 Mar 2023.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [A Corpus for Reasoning About Natural Language Grounded in Photographs](https://mlsanity.com/paper/1811.00491/) (2018)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023)
- [Automatic Spatially-aware Fashion Concept Discovery](https://mlsanity.com/paper/1708.01311/) (2017)
- [Cycle-Consistency for Robust Visual Question Answering](https://mlsanity.com/paper/1902.05660/) (2019)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [Compositional Learning of Image-Text Query for Image Retrieval](https://mlsanity.com/paper/2006.11149/) (2020)
- [Modality-Agnostic Attention Fusion for visual search with text feedback](https://mlsanity.com/paper/2007.00145/) (2020)
- [Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning](https://mlsanity.com/paper/2303.11302/) (2023)
- [Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback](https://mlsanity.com/paper/1905.12794/) (2019)
- [Multimodal Residual Learning for Visual QA](https://mlsanity.com/paper/1606.01455/) (2016)
- [LaSO: Label-Set Operations networks for multi-label few-shot learning](https://mlsanity.com/paper/1902.09811/) (2019)
- [FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks](https://mlsanity.com/paper/2303.02483/) (2023)
- [SAC: Semantic Attention Composition for Text-Conditioned Image Retrieval](https://mlsanity.com/paper/2009.01485/) (2020)
- [CurlingNet: Compositional Learning between Images and Text for Fashion IQ Data](https://mlsanity.com/paper/2003.12299/) (2020)

## Built on by (most cited first, 30 of 43)

- [VISTA: Visualized Text Embedding For Universal Multi-Modal Retrieval](https://mlsanity.com/paper/2406.04292/) (2024)
- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [FineCIR: Explicit Parsing of Fine-Grained Modification Semantics for Composed Image Retrieval](https://mlsanity.com/paper/2503.21309/) (2025)
- [Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encoder](https://mlsanity.com/paper/2305.16304/) (2023)
- [Zero-shot Composed Text-Image Retrieval](https://mlsanity.com/paper/2306.07272/) (2023)
- [CaLa: Complementary Association Learning for Augmenting Composed Image Retrieval](https://mlsanity.com/paper/2405.19149/) (2024)
- [Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negatives](https://mlsanity.com/paper/2404.11317/) (2024)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [TEMA: Anchor the Image, Follow the Text for Multi-Modification Composed Image Retrieval](https://mlsanity.com/paper/2604.21806/) (2026)
- [Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2311.07622/) (2023)
- [Training-free Zero-shot Composed Image Retrieval via Weighted Modality Fusion and Similarity](https://mlsanity.com/paper/2409.04918/) (2024)
- [Instance-Level Composed Image Retrieval](https://mlsanity.com/paper/2510.25387/) (2025)
- [VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering](https://mlsanity.com/paper/2312.12273/) (2023)
- [QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval](https://mlsanity.com/paper/2507.12416/) (2025)
- [Scale Up Composed Image Retrieval Learning via Modification Text Generation](https://mlsanity.com/paper/2504.05316/) (2025)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration](https://mlsanity.com/paper/2403.11373/) (2024)
- [Pseudo-triplet Guided Few-shot Composed Image Retrieval](https://mlsanity.com/paper/2407.06001/) (2024)
- [Composed Image Retrieval for Training-Free Domain Conversion](https://mlsanity.com/paper/2412.03297/) (2024)
- [TMCIR: Token Merge Benefits Composed Image Retrieval](https://mlsanity.com/paper/2504.10995/) (2025)
- [DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrieval](https://mlsanity.com/paper/2505.17796/) (2025)
- [LVLM-empowered Multi-modal Representation Learning for Visual Place Recognition](https://mlsanity.com/paper/2407.06730/) (2024)
- [Modality and Task Adaptation for Enhanced Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2410.23736/) (2024)
- [SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval](https://mlsanity.com/paper/2501.08347/) (2025)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [Automatic Synthesis of High-Quality Triplet Data for Composed Image Retrieval](https://mlsanity.com/paper/2507.05970/) (2025)
- [Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning](https://mlsanity.com/paper/2601.11393/) (2026)
- [Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval](https://mlsanity.com/paper/2603.01082/) (2026)
- [NCL-CIR: Noise-aware Contrastive Learning for Composed Image Retrieval](https://mlsanity.com/paper/2504.04339/) (2025)
- [FACap: A Large-scale Fashion Dataset for Fine-grained Composed Image Retrieval](https://mlsanity.com/paper/2507.07135/) (2025)

## Abstract (arXiv)

Composed image retrieval searches for a target image based on a multi-modal user query comprised of a reference image and modification text describing the desired changes. Existing approaches to solving this challenging task learn a mapping from the (reference image, modification text)-pair to an image embedding that is then matched against a large image corpus. One area that has not yet been explored is the reverse direction, which asks the question, what reference image when modified as described by the text would produce the given target image? In this work we propose a bi-directional training scheme that leverages such reversed queries and can be applied to existing composed image retrieval architectures with minimum changes, which improves the performance of the model. To encode the bi-directional query we prepend a learnable token to the modification text that designates the direction of the query and then finetune the parameters of the text embedding module. We make no other changes to the network architecture. Experiments on two standard datasets show that our novel approach achieves improved performance over a baseline BLIP-based model that itself already achieves competitive performance. Our code is released at https://github.com/Cuberick-Orion/Bi-Blip4CIR.