# End-to-end optimization of goal-driven and visually grounded dialogue systems

arXiv: https://arxiv.org/abs/1703.05423 · HTML: https://mlsanity.com/paper/1703.05423/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2017.

## Built on by (most cited first, 17 of 17)

- [Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning](https://mlsanity.com/paper/2404.19409/) (2024)
- [Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off](https://mlsanity.com/paper/2301.13083/) (2023)
- [Learning Better Visual Dialog Agents with Pretrained Visual-Linguistic Representation](https://mlsanity.com/paper/2105.11541/) (2021)
- [Unified Questioner Transformer for Descriptive Question Generation in Goal-Oriented Visual Dialogue](https://mlsanity.com/paper/2106.15550/) (2021)
- [Saying the Unseen: Video Descriptions via Dialog Agents](https://mlsanity.com/paper/2106.14069/) (2021)
- [Enhancing Visual Dialog Questioner with Entity-based Strategy Learning and Augmented Guesser](https://mlsanity.com/paper/2109.02297/) (2021)
- [I Want This Product but Different : Multimodal Retrieval with Synthetic Query Expansion](https://mlsanity.com/paper/2102.08871/) (2021)
- [LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue](https://mlsanity.com/paper/2210.07777/) (2022)
- [OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog](https://mlsanity.com/paper/2402.13146/) (2024)
- [The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues](https://mlsanity.com/paper/2103.11151/) (2021)
- [Modeling Explicit Concerning States for Reinforcement Learning in Visual Dialogue](https://mlsanity.com/paper/2107.05250/) (2021)
- [Learning Natural Language Generation from Scratch](https://mlsanity.com/paper/2109.09371/) (2021)
- [Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review](https://mlsanity.com/paper/2207.00782/) (2022)
- [Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights](https://mlsanity.com/paper/2207.07255/) (2022)
- [An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games](https://mlsanity.com/paper/2102.00424/) (2021)
- [Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene](https://mlsanity.com/paper/2203.08362/) (2022)
- [VDialogUE: A Unified Evaluation Benchmark for Visually-grounded Dialogue](https://mlsanity.com/paper/2309.07387/) (2023)

## Abstract (arXiv)

End-to-end design of dialogue systems has recently become a popular research topic thanks to powerful tools such as encoder-decoder architectures for sequence-to-sequence learning. Yet, most current approaches cast human-machine dialogue management as a supervised learning problem, aiming at predicting the next utterance of a participant given the full history of the dialogue. This vision is too simplistic to render the intrinsic planning problem inherent to dialogue as well as its grounded nature, making the context of a dialogue larger than the sole history. This is why only chit-chat and question answering tasks have been addressed so far using end-to-end architectures. In this paper, we introduce a Deep Reinforcement Learning method to optimize visually grounded task-oriented dialogues, based on the policy gradient algorithm. This approach is tested on a dataset of 120k dialogues collected through Mechanical Turk and provides encouraging results at solving both the problem of generating natural dialogues and the task of discovering a specific object in a complex picture.