mlsanityData to 7 Oct 2026

Show, Don't Ask: Generative Visual Disambiguation for Composed Image Retrieval with Turn-Valid Coverage

arXiv 2606.18992 ↗First posted 17 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Composed image retrieval (CIR) uses a reference image and a text modification to search for a target image. However, such queries often describe several possible images rather than one exact target, making the user's intent ambiguous. Recent methods address this by using conformal prediction to estimate ambiguity and by asking users clarifying text questions. However, these methods have two limitations: their coverage guarantee only holds at the first interaction, and text questions are often insufficient for resolving fine-grained visual differences such as appearance, attributes, or viewpoint. We propose CLARA, a clarification framework that resolves ambiguity by showing users a small panel of visual alternatives. Instead of answering text questions, the user simply selects the prototype image closest to the intended target. This provides a direct visual signal and avoids relying on a model to predict the user's answer. To maintain valid conformal guarantees across multiple interaction rounds, CLARA reweights calibration using the likelihood ratio induced by the user's selection. The displayed prototypes are also constrained to represent the current candidate set and are snapped to real corpus images, ensuring that generated images cannot artificially improve coverage. Experiments on open-domain and fashion benchmarks show that CLARA matches single-turn state-of-the-art retrieval performance, maintains nominal coverage across interaction rounds, and finds the intended target in fewer rounds than strong text-question baselines. Its advantage is especially clear when ambiguity involves viewpoint or fine-grained attributes, where visual clarification is more effective than textual questioning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

34 of 60 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2017On Calibration of Modern Neural Networkscited by 4,860
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  11. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  12. 2019Conformal Prediction Under Covariate Shiftcited by 511
22 more
  1. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  2. 2020Classification with Valid and Adaptive Coveragecited by 390
  3. 2021Adaptive Conformal Inference Under Distribution Shiftcited by 360
  4. 2017A simple neural network module for relational reasoningcited by 345
  5. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  6. 2018Composing Text and Image for Image Retrieval - An Empirical Odysseycited by 234
  7. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
  8. 2023Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrievalcited by 155
  9. 2018Multilevel Language and Vision Integration for Text-to-Clip Retrievalcited by 128
  10. 2018Dialog-based Interactive Image Retrievalcited by 84
  11. 2023CompoDiff: Versatile Composed Image Retrieval With Latent Diffusioncited by 84
  12. 2022Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuningcited by 54
  13. 2022Conformal Prediction Under Feedback Covariate Shift for Biomolecular Designcited by 53
  14. 2020Modality-Agnostic Attention Fusion for visual search with text feedbackcited by 51
  15. 2020Learning from the Scene and Borrowing from the Rich: Tackling the Long Tail in Scene Graph Generationcited by 22
  16. 2023Towards a Unified Transformer-based Framework for Scene Graph Generation and Human-object Interaction Detectioncited by 15
  17. 2024MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modalitycited by 10
  18. 2026Fixed Anchors Are Not Enough: Dynamic Retrieval and Persistent Homology for Dataset Distillationcited by 10
  19. 2025SPADE: Spatial-Aware Denoising Network for Open-vocabulary Panoptic Scene Graph Generation with Long- and Local-range Context Reasoningcited by 9
  20. 2021Semi-supervised Network Embedding with Differentiable Deep Quantisationcited by 7
  21. 2026Unbiased Dynamic Multimodal Fusioncited by 7
  22. 2019One Network for Multi-Domains: Domain Adaptive Hashing with Intersectant Generative Adversarial Networkcited by 6

Built on by

Not cited yet.