# Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval

arXiv: https://arxiv.org/abs/2309.16137 · HTML: https://mlsanity.com/paper/2309.16137/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks](https://mlsanity.com/paper/2004.06165/) (2020)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [VL-BERT: Pre-training of Generic Visual-Linguistic Representations](https://mlsanity.com/paper/1908.08530/) (2019)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training](https://mlsanity.com/paper/1908.06066/) (2019)
- [ReZero is All You Need: Fast Convergence at Large Depth](https://mlsanity.com/paper/2003.04887/) (2020)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023)
- [CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailment](https://mlsanity.com/paper/2203.07190/) (2022)
- ["This is my unicorn, Fluffy": Personalizing frozen vision-language representations](https://mlsanity.com/paper/2204.01694/) (2022)
- [ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity](https://mlsanity.com/paper/2203.08101/) (2022)
- [Modality-Agnostic Attention Fusion for visual search with text feedback](https://mlsanity.com/paper/2007.00145/) (2020)
- [FashionViL: Fashion-Focused Vision-and-Language Representation Learning](https://mlsanity.com/paper/2207.08150/) (2022)
- [Visualize Before You Write: Imagination-Guided Open-Ended Text Generation](https://mlsanity.com/paper/2210.03765/) (2022)

## Built on by (most cited first, 30 of 52)

- [E5-V: Universal Embeddings with Multimodal Large Language Models](https://mlsanity.com/paper/2407.12580/) (2024)
- [MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions](https://mlsanity.com/paper/2403.19651/) (2024)
- [Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2412.11077/) (2024)
- [Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2401.06825/) (2024)
- [iSEARLE: Improving Textual Inversion for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2405.02951/) (2024)
- [EgoCVR: An Egocentric Benchmark for Fine-Grained Composed Video Retrieval](https://mlsanity.com/paper/2407.16658/) (2024)
- [CoLLM: A Large Language Model for Composed Image Retrieval](https://mlsanity.com/paper/2503.19910/) (2025)
- [Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2503.17109/) (2025)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2311.07622/) (2023)
- [Learning Commonality, Divergence and Variety for Unsupervised Visible-Infrared Person Re-identification](https://mlsanity.com/paper/2402.19026/) (2024)
- [Leveraging Large Vision-Language Model as User Intent-aware Encoder for Composed Image Retrieval](https://mlsanity.com/paper/2412.11087/) (2024)
- [Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Service](https://mlsanity.com/paper/2311.05863/) (2023)
- [CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2312.08648/) (2023)
- [HyCIR: Boosting Zero-Shot Composed Image Retrieval with Synthetic Labels](https://mlsanity.com/paper/2407.05795/) (2024)
- [Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2505.19952/) (2025)
- [A Survey of Multimodal Composite Editing and Retrieval](https://mlsanity.com/paper/2409.05405/) (2024)
- [MLLM-Guided VLM Fine-Tuning with Joint Inference for Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2505.19707/) (2025)
- [Pseudo-triplet Guided Few-shot Composed Image Retrieval](https://mlsanity.com/paper/2407.06001/) (2024)
- [CLIP-SCGI: Synthesized Caption-Guided Inversion for Person Re-Identification](https://mlsanity.com/paper/2410.09382/) (2024)
- [Composed Image Retrieval for Training-Free Domain Conversion](https://mlsanity.com/paper/2412.03297/) (2024)
- [Denoise-I2W: Mapping Images to Denoising Words for Accurate Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2410.17393/) (2024)
- [ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval](https://mlsanity.com/paper/2602.01639/) (2026)
- [An Efficient Post-hoc Framework for Reducing Task Discrepancy of Text Encoders for Composed Image Retrieval](https://mlsanity.com/paper/2406.09188/) (2024)
- [Modality and Task Adaptation for Enhanced Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2410.23736/) (2024)
- [SCOT: Self-Supervised Contrastive Pretraining For Zero-Shot Compositional Retrieval](https://mlsanity.com/paper/2501.08347/) (2025)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)
- [Data-Efficient Generalization for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2503.05204/) (2025)
- [From Mapping to Composing: A Two-Stage Framework for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2504.17990/) (2025)
- [Heterogeneous Uncertainty-Guided Composed Image Retrieval with Fine-Grained Probabilistic Learning](https://mlsanity.com/paper/2601.11393/) (2026)

## Abstract (arXiv)

Different from Composed Image Retrieval task that requires expensive labels for training task-specific models, Zero-Shot Composed Image Retrieval (ZS-CIR) involves diverse tasks with a broad range of visual content manipulation intent that could be related to domain, scene, object, and attribute. The key challenge for ZS-CIR tasks is to learn a more accurate image representation that has adaptive attention to the reference image for various manipulation descriptions. In this paper, we propose a novel context-dependent mapping network, named Context-I2W, for adaptively converting description-relevant Image information into a pseudo-word token composed of the description for accurate ZS-CIR. Specifically, an Intent View Selector first dynamically learns a rotation rule to map the identical image to a task-specific manipulation view. Then a Visual Target Extractor further captures local information covering the main targets in ZS-CIR tasks under the guidance of multiple learnable queries. The two complementary modules work together to map an image to a context-dependent pseudo-word token without extra supervision. Our model shows strong generalization ability on four ZS-CIR tasks, including domain conversion, object composition, object manipulation, and attribute manipulation. It obtains consistent and significant performance boosts ranging from 1.88% to 3.60% over the best methods and achieves new state-of-the-art results on ZS-CIR. Our code is available at https://github.com/Pter61/context-i2w.