# Dual Relation Alignment for Composed Image Retrieval

arXiv: https://arxiv.org/abs/2309.02169 · HTML: https://mlsanity.com/paper/2309.02169/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 5 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [A Corpus for Reasoning About Natural Language Grounded in Photographs](https://mlsanity.com/paper/1811.00491/) (2018)
- [Multi-Grained Vision Language Pre-Training: Aligning Texts with Visual Concepts](https://mlsanity.com/paper/2111.08276/) (2021)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023)
- [Dual-Path Convolutional Image-Text Embeddings with Instance Loss](https://mlsanity.com/paper/1711.05535/) (2017)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [Compositional Learning of Image-Text Query for Image Retrieval](https://mlsanity.com/paper/2006.11149/) (2020)
- [Modality-Agnostic Attention Fusion for visual search with text feedback](https://mlsanity.com/paper/2007.00145/) (2020)
- [Position Focused Attention Network for Image-Text Matching](https://mlsanity.com/paper/1907.09748/) (2019)
- [Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization](https://mlsanity.com/paper/2211.07394/) (2022)
- [Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback](https://mlsanity.com/paper/1905.12794/) (2019)
- [RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network](https://mlsanity.com/paper/2104.03015/) (2021)
- [SAC: Semantic Attention Composition for Text-Conditioned Image Retrieval](https://mlsanity.com/paper/2009.01485/) (2020)
- [CurlingNet: Compositional Learning between Images and Text for Fashion IQ Data](https://mlsanity.com/paper/2003.12299/) (2020)

## Built on by (most cited first, 5 of 5)

- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering](https://mlsanity.com/paper/2312.12273/) (2023)
- [ConText-CIR: Learning from Concepts in Text for Composed Image Retrieval](https://mlsanity.com/paper/2505.20764/) (2025)
- [Composed Image Retrieval for Training-Free Domain Conversion](https://mlsanity.com/paper/2412.03297/) (2024)
- [TMCIR: Token Merge Benefits Composed Image Retrieval](https://mlsanity.com/paper/2504.10995/) (2025)

## Abstract (arXiv)

Composed image retrieval, a task involving the search for a target image using a reference image and a complementary text as the query, has witnessed significant advancements owing to the progress made in cross-modal modeling. Unlike the general image-text retrieval problem with only one alignment relation, i.e., image-text, we argue for the existence of two types of relations in composed image retrieval. The explicit relation pertains to the reference image & complementary text-target image, which is commonly exploited by existing methods. Besides this intuitive relation, the observations during our practice have uncovered another implicit yet crucial relation, i.e., reference image & target image-complementary text, since we found that the complementary text can be inferred by studying the relation between the target image and the reference image. Regrettably, existing methods largely focus on leveraging the explicit relation to learn their networks, while overlooking the implicit relation. In response to this weakness, We propose a new framework for composed image retrieval, termed dual relation alignment, which integrates both explicit and implicit relations to fully exploit the correlations among the triplets. Specifically, we design a vision compositor to fuse reference image and target image at first, then the resulted representation will serve two roles: (1) counterpart for semantic alignment with the complementary text and (2) compensation for the complementary text to boost the explicit relation modeling, thereby implant the implicit relation into the alignment learning. Our method is evaluated on two popular datasets, CIRR and FashionIQ, through extensive experiments. The results confirm the effectiveness of our dual-relation learning in substantially enhancing composed image retrieval performance.