# DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

arXiv: https://arxiv.org/abs/2009.13570 · HTML: https://mlsanity.com/paper/2009.13570/

## Key facts

- Cited by 89 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 30 of 89)

- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processing](https://mlsanity.com/paper/2108.05542/) (2021)
- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injection](https://mlsanity.com/paper/2111.14592/) (2021)
- [Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey](https://mlsanity.com/paper/2503.22458/) (2025)
- [Zero-Shot Dialogue State Tracking via Cross-Task Transfer](https://mlsanity.com/paper/2109.04655/) (2021)
- [In-Context Learning for Few-Shot Dialogue State Tracking](https://mlsanity.com/paper/2203.08568/) (2022)
- [Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning](https://mlsanity.com/paper/2109.06349/) (2021)
- [Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Tracking](https://mlsanity.com/paper/2106.00291/) (2021)
- [In-Context Learning for Text Classification with Many Labels](https://mlsanity.com/paper/2309.10954/) (2023)
- [SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation](https://mlsanity.com/paper/2209.06664/) (2022)
- [DialogBench: Evaluating LLMs as Human-like Dialogue Systems](https://mlsanity.com/paper/2311.01677/) (2023)
- ["How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations](https://mlsanity.com/paper/2109.13489/) (2021)
- [Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System](https://mlsanity.com/paper/2305.02468/) (2023)
- [New Intent Discovery with Pre-training and Contrastive Learning](https://mlsanity.com/paper/2205.12914/) (2022)
- [SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding](https://mlsanity.com/paper/2209.06638/) (2022)
- [Effectiveness of Pre-training for Few-shot Intent Classification](https://mlsanity.com/paper/2109.05782/) (2021)
- [Grounded Entity-Landmark Adaptive Pre-training for Vision-and-Language Navigation](https://mlsanity.com/paper/2308.12587/) (2023)
- [ConvFiT: Conversational Fine-Tuning of Pretrained Language Models](https://mlsanity.com/paper/2109.10126/) (2021)
- [Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information](https://mlsanity.com/paper/2302.05096/) (2023)
- [Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2203.01552/) (2022)
- [ReMeDi: Resources for Multi-domain, Multi-service, Medical Dialogues](https://mlsanity.com/paper/2109.00430/) (2021)
- [Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization](https://mlsanity.com/paper/2205.07208/) (2022)
- [ProtAugment: Unsupervised diverse short-texts paraphrasing for intent detection meta-learning](https://mlsanity.com/paper/2105.12995/) (2021)
- [LAD: Language Models as Data for Zero-Shot Dialog](https://mlsanity.com/paper/2207.14393/) (2022)
- [Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2104.08570/) (2021)
- [ContrastNet: A Contrastive Learning Framework for Few-Shot Text Classification](https://mlsanity.com/paper/2305.09269/) (2023)
- [Rethinking Semi-supervised Learning with Language Models](https://mlsanity.com/paper/2305.13002/) (2023)
- [RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2109.10410/) (2021)
- [Robust Dialogue State Tracking with Weak Supervision and Sparse Data](https://mlsanity.com/paper/2202.03354/) (2022)

## Abstract (arXiv)

A long-standing goal of task-oriented dialogue research is the ability to flexibly adapt dialogue models to new domains. To progress research in this direction, we introduce DialoGLUE (Dialogue Language Understanding Evaluation), a public benchmark consisting of 7 task-oriented dialogue datasets covering 4 distinct natural language understanding tasks, designed to encourage dialogue research in representation-based transfer, domain adaptation, and sample-efficient task learning. We release several strong baseline models, demonstrating performance improvements over a vanilla BERT architecture and state-of-the-art results on 5 out of 7 tasks, by pre-training on a large open-domain dialogue corpus and task-adaptive self-supervised training. Through the DialoGLUE benchmark, the baseline methods, and our evaluation scripts, we hope to facilitate progress towards the goal of developing more general task-oriented dialogue models.