# DIET: Lightweight Language Understanding for Dialogue Systems

arXiv: https://arxiv.org/abs/2004.09936 · HTML: https://mlsanity.com/paper/2004.09936/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2020.

## Built on by (most cited first, 30 of 35)

- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information](https://mlsanity.com/paper/2302.05096/) (2023)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System](https://mlsanity.com/paper/2205.04006/) (2022)
- [ENIGMA-51: Towards a Fine-Grained Understanding of Human-Object Interactions in Industrial Scenarios](https://mlsanity.com/paper/2309.14809/) (2023)
- [Do as I mean, not as I say: Sequence Loss Training for Spoken Language Understanding](https://mlsanity.com/paper/2102.06750/) (2021)
- [DRAGON: A Dialogue-Based Robot for Assistive Navigation with Visual Language Grounding](https://mlsanity.com/paper/2307.06924/) (2023)
- [IntentGPT: Few-shot Intent Discovery with Large Language Models](https://mlsanity.com/paper/2411.10670/) (2024)
- [Natural Language Understanding for Argumentative Dialogue Systems in the Opinion Building Domain](https://mlsanity.com/paper/2103.02691/) (2021)
- [OPERA: Harmonizing Task-Oriented Dialogs and Information Seeking Experience](https://mlsanity.com/paper/2206.12449/) (2022)
- [AI-Enabled Conversational Journaling for Advancing Parkinson's Disease Symptom Tracking](https://mlsanity.com/paper/2503.03532/) (2025)
- [On Safe and Usable Chatbots for Promoting Voter Participation](https://mlsanity.com/paper/2212.11219/) (2022)
- [End-to-End Natural Language Understanding Pipeline for Bangla Conversational Agents](https://mlsanity.com/paper/2107.05541/) (2021)
- [Quick Starting Dialog Systems with Paraphrase Generation](https://mlsanity.com/paper/2204.02546/) (2022)
- [NLU for Game-based Learning in Real: Initial Evaluations](https://mlsanity.com/paper/2205.13754/) (2022)
- [Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic](https://mlsanity.com/paper/2207.00627/) (2022)
- [End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics](https://mlsanity.com/paper/2211.03511/) (2022)
- [PerSHOP -- A Persian dataset for shopping dialogue systems modeling](https://mlsanity.com/paper/2401.00811/) (2024)
- [LSTM-RASA Based Agri Farm Assistant for Farmers](https://mlsanity.com/paper/2204.09717/) (2022)
- [We are all Individuals: The Role of Robot Personality and Human Traits in Trustworthy Interaction](https://mlsanity.com/paper/2307.15568/) (2023)
- [Task-Oriented Dialog Systems for the Senegalese Wolof Language](https://mlsanity.com/paper/2412.11203/) (2024)
- [Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes](https://mlsanity.com/paper/2608.02415/) (2026)
- [COVID-19 Smart Chatbot Prototype for Patient Monitoring](https://mlsanity.com/paper/2103.06816/) (2021)
- [Development of a Conversation State Prediction System](https://mlsanity.com/paper/2107.01462/) (2021)
- [A Sequence to Sequence Model for Extracting Multiple Product Name Entities from Dialog](https://mlsanity.com/paper/2110.14843/) (2021)
- [Multi-stage Clarification in Conversational AI: The case of Question-Answering Dialogue Systems](https://mlsanity.com/paper/2110.15235/) (2021)
- [Inspecting Spoken Language Understanding from Kids for Basic Math Learning at Home](https://mlsanity.com/paper/2306.00482/) (2023)
- [TESS: A Multi-intent Parser for Conversational Multi-Agent Systems with Decentralized Natural Language Understanding Models](https://mlsanity.com/paper/2312.11828/) (2023)
- [CEASEFIRE: An AI-powered system for combatting illicit firearms trafficking](https://mlsanity.com/paper/2406.14949/) (2024)
- [From User Preferences to Optimization Constraints Using Large Language Models](https://mlsanity.com/paper/2503.21360/) (2025)

## Abstract (arXiv)

Large-scale pre-trained language models have shown impressive results on language understanding benchmarks like GLUE and SuperGLUE, improving considerably over other pre-training methods like distributed representations (GloVe) and purely supervised approaches. We introduce the Dual Intent and Entity Transformer (DIET) architecture, and study the effectiveness of different pre-trained representations on intent and entity prediction, two common dialogue language understanding tasks. DIET advances the state of the art on a complex multi-domain NLU dataset and achieves similarly high performance on other simpler datasets. Surprisingly, we show that there is no clear benefit to using large pre-trained models for this task, and in fact DIET improves upon the current state of the art even in a purely supervised setup without any pre-trained embeddings. Our best performing model outperforms fine-tuning BERT and is about six times faster to train.