# A Context-Aware Hierarchical BERT Fusion Network for Multi-turn Dialog Act Detection

arXiv: https://arxiv.org/abs/2109.01267 · HTML: https://mlsanity.com/paper/2109.01267/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 3 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding](https://mlsanity.com/paper/1909.02188/) (2019)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [Zero-shot User Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1809.00385/) (2018)
- [Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems](https://mlsanity.com/paper/1807.11125/) (2018)
- [CM-Net: A Novel Collaborative Memory Network for Spoken Language Understanding](https://mlsanity.com/paper/1909.06937/) (2019)
- [A Contextual Hierarchical Attention Network with Adaptive Objective for Dialogue State Tracking](https://mlsanity.com/paper/2006.01554/) (2020)
- [Composed Variational Natural Language Generation for Few-shot Intents](https://mlsanity.com/paper/2009.10056/) (2020)
- [CASA-NLU: Context-Aware Self-Attentive Natural Language Understanding for Task-Oriented Chatbots](https://mlsanity.com/paper/1909.08705/) (2019)

## Built on by (most cited first, 5 of 5)

- [LARA: Linguistic-Adaptive Retrieval-Augmentation for Multi-Turn Intent Classification](https://mlsanity.com/paper/2403.16504/) (2024)
- [MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU](https://mlsanity.com/paper/2408.08144/) (2024)
- [From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classification](https://mlsanity.com/paper/2411.14252/) (2024)
- [Knowledge Augmented BERT Mutual Network in Multi-turn Spoken Dialogues](https://mlsanity.com/paper/2202.11299/) (2022)
- [Local and Global Contexts for Conversation](https://mlsanity.com/paper/2401.17588/) (2024)

## Abstract (arXiv)

The success of interactive dialog systems is usually associated with the quality of the spoken language understanding (SLU) task, which mainly identifies the corresponding dialog acts and slot values in each turn. By treating utterances in isolation, most SLU systems often overlook the semantic context in which a dialog act is expected. The act dependency between turns is non-trivial and yet critical to the identification of the correct semantic representations. Previous works with limited context awareness have exposed the inadequacy of dealing with complexity in multiproned user intents, which are subject to spontaneous change during turn transitions. In this work, we propose to enhance SLU in multi-turn dialogs, employing a context-aware hierarchical BERT fusion Network (CaBERT-SLU) to not only discern context information within a dialog but also jointly identify multiple dialog acts and slots in each utterance. Experimental results show that our approach reaches new state-of-the-art (SOTA) performances in two complicated multi-turn dialogue datasets with considerable improvements compared with previous methods, which only consider single utterances for multiple intents and slot filling.