Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage
Not on the map. Why?
Abstract (arXiv)
Composed image retrieval (CIR) uses a reference image and a text modification to search for a target image. However, such queries often describe several possible images rather than one exact target, making the user's intent ambiguous. Recent methods address this by using conformal prediction to estimate ambiguity and by asking users clarifying text questions. However, these methods have two limitations: their coverage guarantee only holds at the first interaction, and text questions are often insufficient for resolving fine-grained visual differences such as appearance, attributes, or viewpoint. We propose CLARA, a clarification framework that resolves ambiguity by showing users a small panel of visual alternatives. Instead of answering text questions, the user simply selects the prototype image closest to the intended target. This provides a direct visual signal and avoids relying on a model to predict the user's answer. To maintain valid conformal guarantees across multiple interaction rounds, CLARA reweights calibration using the likelihood ratio induced by the user's selection. The displayed prototypes are also constrained to represent the current candidate set and are snapped to real corpus images, ensuring that generated images cannot artificially improve coverage. Experiments on open-domain and fashion benchmarks show that CLARA matches single-turn state-of-the-art retrieval performance, maintains nominal coverage across interaction rounds, and finds the intended target in fewer rounds than strong text-question baselines. Its advantage is especially clear when ambiguity involves viewpoint or fine-grained attributes, where visual clarification is more effective than textual questioning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
34 of 60 references.
References · 34
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
- 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
- 2019Conformal Prediction Under Covariate Shiftcited by 511
22 more
- 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
- 2020Classification with Valid and Adaptive Coveragecited by 390
- 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
- 2017A simple neural network module for relational reasoningcited by 345
- 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
- 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
- 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
- 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
- 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
- 2018Dialog-based Interactive Image Retrievalcited by 84
- 2023CompoDiff: Versatile Composed Image Retrieval With Latent Diffusioncited by 84
- 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
- 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 53
- 2020Modality-Agnostic Attention Fusion for visual search with text feedbackcited by 51
- 2020Learning from the Scene and Borrowing from the Rich: Tackling the Long Tail in Scene Graph Generationcited by 22
- 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
- 2024MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modalitycited by 10
- 2026Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillationcited by 10
- 2025SPADE: Spatial-Aware Denoising Network for Open-vocabulary Panoptic Scene Graph Generation with Long- and Local-range Context Reasoningcited by 9
- 2021Semi-supervised Network Embedding with Differentiable Deep Quantisationcited by 7
- 2026Unbiased Dynamic Multimodal Fusioncited by 7
- 2019One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networkcited by 6
Built on by
Not cited yet.