mlsanityData to 7 Oct 2026

NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialogue

arXiv 2204.13021 ↗First posted 27 Apr 2022cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

We present NLU++, a novel dataset for natural language understanding (NLU) in task-oriented dialogue (ToD) systems, with the aim to provide a much more challenging evaluation environment for dialogue NLU models, up to date with the current application and industry requirements. NLU++ is divided into two domains (BANKING and HOTELS) and brings several crucial improvements over current commonly used NLU datasets. 1) NLU++ provides fine-grained domain ontologies with a large set of challenging multi-intent sentences, introducing and validating the idea of intent modules that can be combined into complex intents that convey complex user goals, combined with finer-grained and thus more challenging slot sets. 2) The ontology is divided into domain-specific and generic (i.e., domain-universal) intent modules that overlap across domains, promoting cross-domain reusability of annotated examples. 3) The dataset design has been inspired by the problems observed in industrial ToD systems, and 4) it has been collected, filtered and carefully annotated by dialogue NLU experts, yielding high-quality annotated data. Finally, we benchmark a series of current state-of-the-art NLU models on NLU++; the results demonstrate the challenging nature of the dataset, especially in low-data regimes, the validity of `intent modularisation', and call for further research on ToD NLU.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 49 of its 81 references that are in the map. First posted on arXiv 27 Apr 2022.

Builds on

49 of 81 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  8. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  9. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  10. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  11. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  12. 2019Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Datasetcited by 426
37 more
  1. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  2. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 378
  3. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  4. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  5. 2020MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselinescited by 201
  6. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  7. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  8. 2020TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialoguecited by 185
  9. 2019Benchmarking Natural Language Understanding Services for building Conversational Agentscited by 166
  10. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 136
  11. 2020TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Trackingcited by 130
  12. 2019ConveRT: Efficient and Accurate Conversational Representations from Transformerscited by 95
  13. 2018Tensor2Tensor for Neural Machine Translationcited by 85
  14. 2017Frames: A Corpus for Adding Memory to Goal-Oriented Dialogue Systemscited by 84
  15. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 82
  16. 2019Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmaxcited by 58
  17. 2019Training Neural Response Selection for Task-Oriented Dialogue Systemscited by 48
  18. 2021Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuningcited by 46
  19. 2019A Repository of Conversational Datasetscited by 45
  20. 2020Span-ConveRT: Few-shot Span Extraction for Dialog with Pretrained Conversational Representationscited by 42
  21. 2020SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Fillingcited by 42
  22. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  23. 2021Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Trackingcited by 35
  24. 2020Learning to Classify Intents and Slot Labels Given a Handful of Examplescited by 31
  25. 2020LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimizationcited by 30
  26. 2020Language Model is All You Need: Natural Language Understanding as Question Answeringcited by 29
  27. 2019Outlier Detection for Improved Data Quality and Diversity in Dialog Systemscited by 28
  28. 2021Effectiveness of Pre-training for Few-shot Intent Classificationcited by 27
  29. 2020Multi-Domain Dialogue Acts and Response Co-Generationcited by 26
  30. 2020Example-Driven Intent Prediction with Observerscited by 26
  31. 2020ConVEx: Data-Efficient and Few-Shot Slot Labelingcited by 22
  32. 2016Conversational Contextual Cues: The Case of Personalization and History for Response Rankingcited by 21
  33. 2020Few-shot Learning for Multi-label Intent Detectioncited by 20
  34. 2020Probing Task-Oriented Dialogue Representation from Language Modelscited by 15
  35. 2022Improved and Efficient Conversational Slot Labeling through Question Answeringcited by 10
  36. 2017A Benchmarking Environment for Reinforcement Learning Based Task Oriented Dialogue Managementcited by 8
  37. 2019Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systemscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 2; 2025: 1; 2026: 1.

8 papers.

  1. 2024Generative AI for Synthetic Data Generation: Methods, Challenges and the Futurecited by 31
  2. 2023DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AIcited by 21
  3. 2023Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systemscited by 16
  4. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  5. 2024Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?cited by 11
  6. 2023Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Datacited by 3
  7. 2026IntentGrasp: A Comprehensive Benchmark for Intent Understandingcited by 1
  8. 2025Multi-Intent Spoken Language Understanding: Methods, Trends, and Challengescited by 0