# Modality-Agnostic Attention Fusion for visual search with text feedback

arXiv: https://arxiv.org/abs/2007.00145 · HTML: https://mlsanity.com/paper/2007.00145/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2020.

## Built on by (most cited first, 30 of 51)

- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023)
- [CompoDiff: Versatile Composed Image Retrieval With Latent Diffusion](https://mlsanity.com/paper/2303.11916/) (2023)
- [Kaleido-BERT: Vision-Language Pre-training on Fashion Domain](https://mlsanity.com/paper/2103.16110/) (2021)
- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [Composed Image Retrieval using Contrastive Learning and Task-oriented CLIP-based Features](https://mlsanity.com/paper/2308.11485/) (2023)
- [CoVR-2: Automatic Data Construction for Composed Video Retrieval](https://mlsanity.com/paper/2308.14746/) (2023)
- [Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2309.16137/) (2023)
- [Data Roaming and Quality Assessment for Composed Image Retrieval](https://mlsanity.com/paper/2303.09429/) (2023)
- [Bi-directional Training for Composed Image Retrieval via Text Prompt Learning](https://mlsanity.com/paper/2303.16604/) (2023)
- [Knowledge-Enhanced Dual-stream Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2403.16005/) (2024)
- [Candidate Set Re-ranking for Composed Image Retrieval with Dual Multi-modal Encoder](https://mlsanity.com/paper/2305.16304/) (2023)
- [RTIC: Residual Learning for Text and Image Composition using Graph Convolutional Network](https://mlsanity.com/paper/2104.03015/) (2021)
- [Language-only Efficient Training of Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2312.01998/) (2023)
- [M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining](https://mlsanity.com/paper/2109.04275/) (2021)
- [Composed Video Retrieval via Enriched Context and Discriminative Embeddings](https://mlsanity.com/paper/2403.16997/) (2024)
- [Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negatives](https://mlsanity.com/paper/2404.11317/) (2024)
- [Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxy](https://mlsanity.com/paper/2411.16752/) (2024)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering](https://mlsanity.com/paper/2312.12273/) (2023)
- [QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval](https://mlsanity.com/paper/2507.12416/) (2025)
- [Image Search with Text Feedback by Additive Attention Compositional Learning](https://mlsanity.com/paper/2203.03809/) (2022)
- [Scale Up Composed Image Retrieval Learning via Modification Text Generation](https://mlsanity.com/paper/2504.05316/) (2025)
- [UIGR: Unified Interactive Garment Retrieval](https://mlsanity.com/paper/2204.03111/) (2022)
- [Language Guided Local Infiltration for Interactive Image Retrieval](https://mlsanity.com/paper/2304.07747/) (2023)
- [Beyond Simple Edits: Composed Video Retrieval with Dense Modifications](https://mlsanity.com/paper/2508.14039/) (2025)
- [Training and challenging models for text-guided fashion image retrieval](https://mlsanity.com/paper/2204.11004/) (2022)
- [Ranking-aware Uncertainty for Text-guided Image Retrieval](https://mlsanity.com/paper/2308.08131/) (2023)
- [Dual Relation Alignment for Composed Image Retrieval](https://mlsanity.com/paper/2309.02169/) (2023)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)

## Abstract (arXiv)

Image retrieval with natural language feedback offers the promise of catalog search based on fine-grained visual features that go beyond objects and binary attributes, facilitating real-world applications such as e-commerce. Our Modality-Agnostic Attention Fusion (MAAF) model combines image and text features and outperforms existing approaches on two visual search with modifying phrase datasets, Fashion IQ and CSS, and performs competitively on a dataset with only single-word modifications, Fashion200k. We also introduce two new challenging benchmarks adapted from Birds-to-Words and Spot-the-Diff, which provide new settings with rich language inputs, and we show that our approach without modification outperforms strong baselines. To better understand our model, we conduct detailed ablations on Fashion IQ and provide visualizations of the surprising phenomenon of words avoiding "attending" to the image region they refer to.