mlsanityData to 7 Oct 2026

MULTI3NLU++: A Multilingual, Multi-Intent, Multi-Domain Dataset for Natural Language Understanding in Task-Oriented Dialogue

arXiv 2212.10455 ↗First posted 20 Dec 2022cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Task-oriented dialogue (TOD) systems have been widely deployed in many industries as they deliver more efficient customer support. These systems are typically constructed for a single domain or language and do not generalise well beyond this. To support work on Natural Language Understanding (NLU) in TOD across multiple languages and domains simultaneously, we constructed MULTI3NLU++, a multilingual, multi-intent, multi-domain dataset. MULTI3NLU++ extends the English only NLU++ dataset to include manual translations into a range of high, medium, and low resource languages (Spanish, Marathi, Turkish and Amharic), in two domains (BANKING and HOTELS). Because of its multi-intent property, MULTI3NLU++ represents complex and natural user goals, and therefore allows us to measure the realistic performance of TOD systems in a varied set of the world's languages. We use MULTI3NLU++ to benchmark state-of-the-art multilingual models for the NLU tasks of intent detection and slot labelling for TOD systems in the multilingual setting. The results demonstrate the challenging nature of the dataset, particularly in the low-resource language setting, offering ample room for future experimentation in multi-domain multilingual TOD setups.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 47 of its 68 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

47 of 68 references.

References · 47

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  8. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  9. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  11. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  12. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
35 more
  1. 2020Beyond English-Centric Multilingual Machine Translationcited by 559
  2. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  3. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 419
  4. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  5. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  6. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  7. 2021Visually Grounded Reasoning across Languages and Culturescited by 188
  8. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  9. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 167
  10. 2021MasakhaNER: Named Entity Recognition for African Languagescited by 155
  11. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 142
  12. 2018Cross-Lingual Transfer Learning for Multilingual Task Oriented Dialogcited by 124
  13. 2020Low-resource Languages: A Review of Past Work and Future Challengescited by 90
  14. 2020End-to-End Slot Alignment and Recognition for Cross-Lingual NLUcited by 82
  15. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  16. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  17. 2021AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languagescited by 50
  18. 2021Discrete and Soft Prompting for Multilingual Modelscited by 49
  19. 2021From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understandingcited by 43
  20. 2021Multilingual and Cross-Lingual Intent Detection from Spoken Datacited by 35
  21. 2021GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systemscited by 30
  22. 2022Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialogcited by 30
  23. 2019Post-editese: an Exacerbated Translationesecited by 29
  24. 2020Language Model is All You Need: Natural Language Understanding as Question Answeringcited by 29
  25. 2022Cross-Lingual Dialogue Dataset Creation via Outline-Based Generationcited by 23
  26. 2021MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learningcited by 21
  27. 2021Intent Detection and Slot Filling for Vietnamesecited by 19
  28. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 16
  29. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  30. 2019Almawave-SLU: A new dataset for SLU in Italiancited by 13
  31. 2021Cross-lingual Intermediate Fine-tuning improves Dialogue State Trackingcited by 11
  32. 2021AllWOZ: Towards Multilingual Task-Oriented Dialog Systems for Allcited by 9
  33. 2018An Ontology-Based Dialogue Management System for Banking and Finance Dialogue Systemscited by 7
  34. 2022Zero-shot Cross-lingual Transfer is Under-specified Optimizationcited by 4
  35. 2019Real-world Conversational AI for Hotel Bookingscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 7; 2024: 5; 2025: 3.

15 papers.

Sort:

By area:

  1. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  2. 2023Romanization-based Large-scale Adaptation of Multilingual Language Modelscited by 21
  3. 2024Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyondcited by 20
  4. 2023Multi3WOZ: A Multilingual, Multi-Domain, Multi-Parallel Dataset for Training and Evaluating Culturally Adapted Task-Oriented Dialog Systemscited by 16
  5. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 15
  6. 2024The Zeno's Paradox of `Low-Resource' Languagescited by 12
  7. 2024Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?cited by 11
  8. 2024When Every Token Counts: Optimal Segmentation for Low-Resource Language Modelscited by 8
  9. 2024M2QA: Multi-domain Multilingual Question Answeringcited by 5
  10. 2023A Systematic Study of Performance Disparities in Multilingual Task-Oriented Dialogue Systemscited by 4
  11. 2023SQATIN: Supervised Instruction Tuning Meets Question Answering for Improved Dialogue NLUcited by 3
  12. 2023OpenSLU: A Unified, Modularized, and Extensible Toolkit for Spoken Language Understandingcited by 2
3 more
  1. 2023DiagGPT: An LLM-based and Multi-agent Dialogue System with Automatic Topic Management for Flexible Task-Oriented Dialoguecited by 2
  2. 2025WolBanking77: Wolof Banking Speech Intent Classification Datasetcited by 0
  3. 2025Multi-Intent Spoken Language Understanding: Methods, Trends, and Challengescited by 0