# Improving Context Modelling in Multimodal Dialogue Generation

arXiv: https://arxiv.org/abs/1810.11955 · HTML: https://mlsanity.com/paper/1810.11955/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2018.

## Built on by (most cited first, 3 of 3)

- [Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generation](https://mlsanity.com/paper/2408.11849/) (2024)
- [A Unified Framework for Slot based Response Generation in a Multimodal Dialogue System](https://mlsanity.com/paper/2305.17433/) (2023)
- [From Knowledge Augmentation to Multi-tasking: Towards Human-like Dialogue Systems](https://mlsanity.com/paper/2212.03279/) (2022)

## Abstract (arXiv)

In this work, we investigate the task of textual response generation in a multimodal task-oriented dialogue system. Our work is based on the recently released Multimodal Dialogue (MMD) dataset (Saha et al., 2017) in the fashion domain. We introduce a multimodal extension to the Hierarchical Recurrent Encoder-Decoder (HRED) model and show that this extension outperforms strong baselines in terms of text-based similarity metrics. We also showcase the shortcomings of current vision and language models by performing an error analysis on our system's output.