# A Knowledge-Grounded Multimodal Search-Based Conversational Agent

arXiv: https://arxiv.org/abs/1810.11954 · HTML: https://mlsanity.com/paper/1810.11954/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2018.

## Built on by (most cited first, 5 of 5)

- [Simple Dialogue System with AUDITED](https://mlsanity.com/paper/2110.11881/) (2021)
- [A Survey on Conversational Search and Applications in Biomedicine](https://mlsanity.com/paper/2211.15328/) (2022)
- [Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems](https://mlsanity.com/paper/2212.09252/) (2022)
- [A Unified Framework for Slot based Response Generation in a Multimodal Dialogue System](https://mlsanity.com/paper/2305.17433/) (2023)
- [Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Survey](https://mlsanity.com/paper/2411.09166/) (2024)

## Abstract (arXiv)

Multimodal search-based dialogue is a challenging new task: It extends visually grounded question answering systems into multi-turn conversations with access to an external database. We address this new challenge by learning a neural response generation system from the recently released Multimodal Dialogue (MMD) dataset (Saha et al., 2017). We introduce a knowledge-grounded multimodal conversational model where an encoded knowledge base (KB) representation is appended to the decoder input. Our model substantially outperforms strong baselines in terms of text-based similarity measures (over 9 BLEU points, 3 of which are solely due to the use of additional information from the KB.