# Improving Generative Visual Dialog by Answering Diverse Questions

arXiv: https://arxiv.org/abs/1909.10470 · HTML: https://mlsanity.com/paper/1909.10470/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2019.

## Built on by (most cited first, 21 of 21)

- [CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendation](https://mlsanity.com/paper/2403.06447/) (2024)
- [Chatting Makes Perfect: Chat-based Image Retrieval](https://mlsanity.com/paper/2305.20062/) (2023)
- [MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding](https://mlsanity.com/paper/2502.15786/) (2025)
- [ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoning](https://mlsanity.com/paper/2503.10166/) (2025)
- [EasyGen: Easing Multimodal Generation with BiDiffuser and LLMs](https://mlsanity.com/paper/2310.08949/) (2023)
- [VD-PCR: Improving Visual Dialog with Pronoun Coreference Resolution](https://mlsanity.com/paper/2205.14693/) (2022)
- [Detours for Navigating Instructional Videos](https://mlsanity.com/paper/2401.01823/) (2024)
- [A Metamodel and Framework for Artificial General Intelligence From Theory to Practice](https://mlsanity.com/paper/2102.06112/) (2021)
- [Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser](https://mlsanity.com/paper/2109.02297/) (2021)
- [Looking for Confirmations: An Effective and Human-Like Visual Dialogue Strategy](https://mlsanity.com/paper/2109.05312/) (2021)
- [Affective Visual Dialog: A Large-Scale Benchmark for Emotional Reasoning Based on Visually Grounded Conversations](https://mlsanity.com/paper/2308.16349/) (2023)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [COCO is "ALL'' You Need for Visual Instruction Fine-tuning](https://mlsanity.com/paper/2401.08968/) (2024)
- [The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues](https://mlsanity.com/paper/2103.11151/) (2021)
- [Modeling Explicit Concerning States for Reinforcement Learning in Visual Dialogue](https://mlsanity.com/paper/2107.05250/) (2021)
- [Can Visual Dialogue Models Do Scorekeeping? Exploring How Dialogue Representations Incrementally Encode Shared Knowledge](https://mlsanity.com/paper/2204.06970/) (2022)
- [Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review](https://mlsanity.com/paper/2207.00782/) (2022)
- [Overprotective Training Environments Fall Short at Testing Time: Let Models Contribute to Their Own Training](https://mlsanity.com/paper/2103.11145/) (2021)
- [Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene](https://mlsanity.com/paper/2203.08362/) (2022)
- [Multi-Modal Dialogue State Tracking for Playing GuessWhich Game](https://mlsanity.com/paper/2408.08431/) (2024)
- [Multimodal Multihop Source Retrieval for Web Question Answering](https://mlsanity.com/paper/2501.04173/) (2025)

## Abstract (arXiv)

Prior work on training generative Visual Dialog models with reinforcement learning(Das et al.) has explored a Qbot-Abot image-guessing game and shown that this 'self-talk' approach can lead to improved performance at the downstream dialog-conditioned image-guessing task. However, this improvement saturates and starts degrading after a few rounds of interaction, and does not lead to a better Visual Dialog model. We find that this is due in part to repeated interactions between Qbot and Abot during self-talk, which are not informative with respect to the image. To improve this, we devise a simple auxiliary objective that incentivizes Qbot to ask diverse questions, thus reducing repetitions and in turn enabling Abot to explore a larger state space during RL ie. be exposed to more visual concepts to talk about, and varied questions to answer. We evaluate our approach via a host of automatic metrics and human studies, and demonstrate that it leads to better dialog, ie. dialog that is more diverse (ie. less repetitive), consistent (ie. has fewer conflicting exchanges), fluent (ie. more human-like),and detailed, while still being comparably image-relevant as prior work and ablations.