# Topic-Aware Response Generation in Task-Oriented Dialogue with Unstructured Knowledge Access

arXiv: https://arxiv.org/abs/2212.05373 · HTML: https://mlsanity.com/paper/2212.05373/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 10 Dec 2022.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Knowledge-Grounded Neural Conversation Model](https://mlsanity.com/paper/1702.01932/) (2017)
- [Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1907.05774/) (2019)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [doc2dial: A Goal-Oriented Document-Grounded Dialogue Dataset](https://mlsanity.com/paper/2011.06623/) (2020)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access](https://mlsanity.com/paper/2006.03533/) (2020)
- [A Controllable Model of Grounded Response Generation](https://mlsanity.com/paper/2005.00613/) (2020)
- [Overview of the Ninth Dialog System Technology Challenge: DSTC9](https://mlsanity.com/paper/2011.06486/) (2020)
- [Topic-Aware Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2009.12539/) (2020)
- [A Sequence-to-Sequence Approach to Dialogue State Tracking](https://mlsanity.com/paper/2011.09553/) (2020)
- [Zero-Resource Knowledge-Grounded Dialogue Generation](https://mlsanity.com/paper/2008.12918/) (2020)
- [Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading](https://mlsanity.com/paper/1906.02738/) (2019)
- [Dually Interactive Matching Network for Personalized Response Selection in Retrieval-Based Chatbots](https://mlsanity.com/paper/1908.05859/) (2019)
- [DeepCopy: Grounded Response Generation with Hierarchical Pointer Networks](https://mlsanity.com/paper/1908.10731/) (2019)
- [Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2204.06677/) (2022)
- [Improving Knowledge-aware Dialogue Generation via Knowledge Base Question Answering](https://mlsanity.com/paper/1912.07491/) (2019)
- [Modeling Topical Relevance for Multi-Turn Dialogue Generation](https://mlsanity.com/paper/2009.12735/) (2020)
- [A Document-grounded Matching Network for Response Selection in Retrieval-based Chatbots](https://mlsanity.com/paper/1906.04362/) (2019)
- [Thinking Globally, Acting Locally: Distantly Supervised Global-to-Local Knowledge Selection for Background Based Conversation](https://mlsanity.com/paper/1908.09528/) (2019)
- [Filtering before Iteratively Referring for Knowledge-Grounded Response Selection in Retrieval-Based Chatbots](https://mlsanity.com/paper/2004.14550/) (2020)
- [ASSIST: Towards Label Noise-Robust Dialogue State Tracking](https://mlsanity.com/paper/2202.13024/) (2022)
- [CAiRE in DialDoc21: Data Augmentation for Information-Seeking Dialogue System](https://mlsanity.com/paper/2106.03530/) (2021)

## Built on by (most cited first, 3 of 3)

- [Schema-Guided User Satisfaction Modeling for Task-Oriented Dialogues](https://mlsanity.com/paper/2305.16798/) (2023)
- [CAUSE: Counterfactual Assessment of User Satisfaction Estimation in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2403.19056/) (2024)
- [Beyond Continuity: Challenges of Context Switching in Multi-Turn Dialogue with LLMs](https://mlsanity.com/paper/2605.09268/) (2026)

## Abstract (arXiv)

To alleviate the problem of structured databases' limited coverage, recent task-oriented dialogue systems incorporate external unstructured knowledge to guide the generation of system responses. However, these usually use word or sentence level similarities to detect the relevant knowledge context, which only partially capture the topical level relevance. In this paper, we examine how to better integrate topical information in knowledge grounded task-oriented dialogue and propose ``Topic-Aware Response Generation'' (TARG), an end-to-end response generation model. TARG incorporates multiple topic-aware attention mechanisms to derive the importance weighting scheme over dialogue utterances and external knowledge sources towards a better understanding of the dialogue history. Experimental results indicate that TARG achieves state-of-the-art performance in knowledge selection and response generation, outperforming previous state-of-the-art by 3.2, 3.6, and 4.2 points in EM, F1 and BLEU-4 respectively on Doc2Dial, and performing comparably with previous work on DSTC9; both being knowledge-grounded task-oriented dialogue datasets.