# Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy

arXiv: https://arxiv.org/abs/2109.05312 · HTML: https://mlsanity.com/paper/2109.05312/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 11 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [GuessWhat?! Visual object discovery through multi-modal dialogue](https://mlsanity.com/paper/1611.08481/) (2016)
- [Sequence-to-Sequence Generation for Spoken Dialogue via Deep Syntax Trees and Strings](https://mlsanity.com/paper/1606.05491/) (2016)
- [Improving Generative Visual Dialog by Answering Diverse Questions](https://mlsanity.com/paper/1909.10470/) (2019)
- [Beyond task success: A closer look at jointly learning to see, ask, and GuessWhat](https://mlsanity.com/paper/1809.03408/) (2018)
- [Leveraging Sentence Similarity in Natural Language Generation: Improving Beam Search using Range Voting](https://mlsanity.com/paper/1908.06288/) (2019)
- [What Should I Ask? Using Conversationally Informative Rewards for Goal-Oriented Visual Dialog](https://mlsanity.com/paper/1907.12021/) (2019)
- [Visual Dialogue State Tracking for Question Generation](https://mlsanity.com/paper/1911.07928/) (2019)
- [Incremental Beam Manipulation for Natural Language Generation](https://mlsanity.com/paper/2102.02574/) (2021)
- [Char2char Generation with Reranking for the E2E NLG Challenge](https://mlsanity.com/paper/1811.05826/) (2018)
- [The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues](https://mlsanity.com/paper/2103.11151/) (2021)

## Built on by (most cited first, 4 of 4)

- [Asking the Right Question at the Right Time: Human and Model Uncertainty Guidance to Ask Clarification Questions](https://mlsanity.com/paper/2402.06509/) (2024)
- [Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain](https://mlsanity.com/paper/2406.17453/) (2024)
- [Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review](https://mlsanity.com/paper/2207.00782/) (2022)
- [Are Current Decoding Strategies Capable of Facing the Challenges of Visual Dialogue?](https://mlsanity.com/paper/2210.12997/) (2022)

## Abstract (arXiv)

Generating goal-oriented questions in Visual Dialogue tasks is a challenging and long-standing problem. State-Of-The-Art systems are shown to generate questions that, although grammatically correct, often lack an effective strategy and sound unnatural to humans. Inspired by the cognitive literature on information search and cross-situational word learning, we design Confirm-it, a model based on a beam search re-ranking algorithm that guides an effective goal-oriented strategy by asking questions that confirm the model's conjecture about the referent. We take the GuessWhat?! game as a case-study. We show that dialogues generated by Confirm-it are more natural and effective than beam search decoding without re-ranking.