# Knowledge Augmented BERT Mutual Network in Multi-turn Spoken Dialogues

arXiv: https://arxiv.org/abs/2202.11299 · HTML: https://mlsanity.com/paper/2202.11299/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 23 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [Knowledge-Grounded Dialogue Generation with Pre-trained Language Models](https://mlsanity.com/paper/2010.08824/) (2020)
- [A Bi-model based RNN Semantic Frame Parsing Model for Intent Detection and Slot Filling](https://mlsanity.com/paper/1812.10235/) (2018)
- [Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems](https://mlsanity.com/paper/1807.11125/) (2018)
- [CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots](https://mlsanity.com/paper/1909.08705/) (2019)
- [A Context-Aware Hierarchical BERT Fusion Network for Multi-turn Dialog Act Detection](https://mlsanity.com/paper/2109.01267/) (2021)

## Abstract (arXiv)

Modern spoken language understanding (SLU) systems rely on sophisticated semantic notions revealed in single utterances to detect intents and slots. However, they lack the capability of modeling multi-turn dynamics within a dialogue particularly in long-term slot contexts. Without external knowledge, depending on limited linguistic legitimacy within a word sequence may overlook deep semantic information across dialogue turns. In this paper, we propose to equip a BERT-based joint model with a knowledge attention module to mutually leverage dialogue contexts between two SLU tasks. A gating mechanism is further utilized to filter out irrelevant knowledge triples and to circumvent distracting comprehension. Experimental results in two complicated multi-turn dialogue datasets have demonstrate by mutually modeling two SLU tasks with filtered knowledge and dialogue contexts, our approach has considerable improvements compared with several competitive baselines.