# Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLU

arXiv: https://arxiv.org/abs/1910.00912 · HTML: https://mlsanity.com/paper/1910.00912/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 13 of 13)

- [Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System](https://mlsanity.com/paper/2205.04006/) (2022)
- [ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarios](https://mlsanity.com/paper/2309.14809/) (2023)
- [Intent Identification and Entity Extraction for Healthcare Queries in Indic Languages](https://mlsanity.com/paper/2302.09685/) (2023)
- [Interactive Storytelling for Children: A Case-study of Design and Development Considerations for Ethical Conversational AI](https://mlsanity.com/paper/2107.13076/) (2021)
- [Controlled Cue Generation for Play Scripts](https://mlsanity.com/paper/2112.06953/) (2021)
- [Introducing Semantics into Speech Encoders](https://mlsanity.com/paper/2211.08402/) (2022)
- [ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions](https://mlsanity.com/paper/2106.07999/) (2021)
- [NLU for Game-based Learning in Real: Initial Evaluations](https://mlsanity.com/paper/2205.13754/) (2022)
- [End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics](https://mlsanity.com/paper/2211.03511/) (2022)
- [Getting the Most out of Simile Recognition](https://mlsanity.com/paper/2211.05984/) (2022)
- [Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks](https://mlsanity.com/paper/2212.08489/) (2022)
- [Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home](https://mlsanity.com/paper/2306.00482/) (2023)
- [Leveraging Information Retrieval to Enhance Spoken Language Understanding Prompts in Few-Shot Learning](https://mlsanity.com/paper/2506.03035/) (2025)

## Abstract (arXiv)

We present a new neural architecture for wide-coverage Natural Language Understanding in Spoken Dialogue Systems. We develop a hierarchical multi-task architecture, which delivers a multi-layer representation of sentence meaning (i.e., Dialogue Acts and Frame-like structures). The architecture is a hierarchy of self-attention mechanisms and BiLSTM encoders followed by CRF tagging layers. We describe a variety of experiments, showing that our approach obtains promising results on a dataset annotated with Dialogue Acts and Frame Semantics. Moreover, we demonstrate its applicability to a different, publicly available NLU dataset annotated with domain-specific intents and corresponding semantic roles, providing overall performance higher than state-of-the-art tools such as RASA, Dialogflow, LUIS, and Watson. For example, we show an average 4.45% improvement in entity tagging F-score over Rasa, Dialogflow and LUIS.