# Pseudo-triplet Guided Few-shot Composed Image Retrieval

arXiv: https://arxiv.org/abs/2407.06001 · HTML: https://mlsanity.com/paper/2407.06001/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 53 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 30 of 31)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [A Corpus for Reasoning About Natural Language Grounded in Photographs](https://mlsanity.com/paper/1811.00491/) (2018)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Models](https://mlsanity.com/paper/2108.04024/) (2021)
- [Composing Text and Image for Image Retrieval - An Empirical Odyssey](https://mlsanity.com/paper/1812.07119/) (2018)
- [Zero-Shot Composed Image Retrieval with Textual Inversion](https://mlsanity.com/paper/2303.15247/) (2023)
- [Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2302.03084/) (2023)
- [Vision-by-Language for Training-Free Compositional Image Retrieval](https://mlsanity.com/paper/2310.09291/) (2023)
- [Learning Active Learning from Data](https://mlsanity.com/paper/1703.03365/) (2017)
- [CompoDiff: Versatile Composed Image Retrieval With Latent Diffusion](https://mlsanity.com/paper/2303.11916/) (2023)
- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [Compositional Learning of Image-Text Query for Image Retrieval](https://mlsanity.com/paper/2006.11149/) (2020)
- [Context-I2W: Mapping Images to Context-dependent Words for Accurate Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2309.16137/) (2023)
- [Bi-directional Training for Composed Image Retrieval via Text Prompt Learning](https://mlsanity.com/paper/2303.16604/) (2023)
- [Neural Naturalist: Generating Fine-Grained Image Comparisons](https://mlsanity.com/paper/1909.04101/) (2019)
- [Target-Guided Composed Image Retrieval](https://mlsanity.com/paper/2309.01366/) (2023)
- [Simple but Effective Raw-Data Level Multimodal Fusion for Composed Image Retrieval](https://mlsanity.com/paper/2404.15875/) (2024)
- [Training-free Zero-shot Composed Image Retrieval with Local Concept Reranking](https://mlsanity.com/paper/2312.08924/) (2023)
- [Self-Training Boosted Multi-Factor Matching Network for Composed Image Retrieval](https://mlsanity.com/paper/2305.09979/) (2023)
- [Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrieval](https://mlsanity.com/paper/2203.03853/) (2022)
- [Neural Compatibility Modeling with Attentive Knowledge Distillation](https://mlsanity.com/paper/1805.00313/) (2018)
- [Adaptive Superpixel for Active Learning in Semantic Segmentation](https://mlsanity.com/paper/2303.16817/) (2023)
- [Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2311.07622/) (2023)
- [HS-GCN: Hamming Spatial Graph Convolutional Networks for Recommendation](https://mlsanity.com/paper/2301.05430/) (2023)
- [Learnable Pillar-based Re-ranking for Image-Text Retrieval](https://mlsanity.com/paper/2304.12570/) (2023)
- [Multi-queue Momentum Contrast for Microvideo-Product Retrieval](https://mlsanity.com/paper/2212.11471/) (2022)
- [Dual Semantic Knowledge Composed Multimodal Dialog Systems](https://mlsanity.com/paper/2305.09990/) (2023)

## Built on by (most cited first, 4 of 4)

- [Missing Target-Relevant Information Prediction with World Model for Accurate Zero-Shot Composed Image Retrieval](https://mlsanity.com/paper/2503.17109/) (2025)
- [A Comprehensive Survey on Composed Image Retrieval](https://mlsanity.com/paper/2502.18495/) (2025)
- [DetailFusion: A Dual-branch Framework with Detail Enhancement for Composed Image Retrieval](https://mlsanity.com/paper/2505.17796/) (2025)
- [Composed Multi-modal Retrieval: A Survey of Approaches and Applications](https://mlsanity.com/paper/2503.01334/) (2025)

## Abstract (arXiv)

Composed Image Retrieval (CIR) is a challenging task that aims to retrieve the target image with a multimodal query, i.e., a reference image, and its complementary modification text. As previous supervised or zero-shot learning paradigms all fail to strike a good trade-off between the model's generalization ability and retrieval performance, recent researchers have introduced the task of few-shot CIR (FS-CIR) and proposed a textual inversion-based network based on pretrained CLIP model to realize it. Despite its promising performance, the approach encounters two key limitations: simply relying on the few annotated samples for CIR model training and indiscriminately selecting training triplets for CIR model fine-tuning. To address these two limitations, we propose a novel two-stage pseudo triplet guided few-shot CIR scheme, dubbed PTG-FSCIR. In the first stage, we propose an attentive masking and captioning-based pseudo triplet generation method, to construct pseudo triplets from pure image data and use them to fulfill the CIR-task specific pertaining. In the second stage, we propose a challenging triplet-based CIR fine-tuning method, where we design a pseudo modification text-based sample challenging score estimation strategy and a robust top range-based random sampling strategy for sampling robust challenging triplets to promote the model fine-tuning. Notably, our scheme is plug-and-play and compatible with any existing supervised CIR models. We test our scheme across two backbones on three public datasets (i.e., FashionIQ, CIRR, and Birds-to-Words), achieving maximum improvements of 13.3%, 22.2%, and 17.4% respectively, demonstrating our scheme's efficacy.