mlsanityData to 14 Oct 2026

MDIA: A Benchmark for Multilingual Dialogue Generation in 46 Languages

arXiv 2208.13078 ↗First posted 27 Aug 2022cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Owing to the lack of corpora for low-resource languages, current works on dialogue generation have mainly focused on English. In this paper, we present mDIA, the first large-scale multilingual benchmark for dialogue generation across low- to high-resource languages. It covers real-life conversations in 46 languages across 19 language families. We present baseline results obtained by fine-tuning the multilingual, non-dialogue-focused pre-trained model mT5 as well as English-centric, dialogue-focused pre-trained chatbot DialoGPT. The results show that mT5-based models perform better on sacreBLEU and BertScore but worse on diversity. Even though promising results are found in few-shot and zero-shot scenarios, there is a large gap between the generation quality in English and other languages. We hope that the release of mDIA could encourage more works on multilingual dialogue generation to promote language diversity.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 27 Aug 2022.

Builds on

42 of 55 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  8. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  9. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  10. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  11. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  12. 2020Recipes for building an open-domain chatbotcited by 593
30 more
  1. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  2. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  3. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 347
  4. 2019MLQA: Evaluating Cross-lingual Extractive Question Answeringcited by 340
  5. 2020XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generationcited by 222
  6. 2018Marian: Fast Neural Machine Translation in C++cited by 203
  7. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  8. 2021XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluationcited by 168
  9. 2016Dual Learning for Machine Translationcited by 148
  10. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  11. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  12. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 116
  13. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
  14. 2020CrossWOZ: A Large-Scale Chinese Cross-Domain Task-Oriented Dialogue Datasetcited by 66
  15. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  16. 2020Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognitioncited by 43
  17. 2020XPersona: Evaluating Multilingual Personalized Chatbotcited by 36
  18. 2021GlobalWoZ: Globalizing MultiWoZ to Develop Multilingual Task-Oriented Dialogue Systemscited by 31
  19. 2021Cross-Lingual Abstractive Summarization with Limited Parallel Resourcescited by 27
  20. 2020It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Informationcited by 26
  21. 2021Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systemscited by 19
  22. 2021Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systemscited by 17
  23. 2020Multilingual AMR-to-Text Generationcited by 12
  24. 2021Smaaprat: DialoGPT for Natural Language Generation of Swedish Dialogue by Transfer Learningcited by 11
  25. 2021A Dataset and Baselines for Multilingual Reply Suggestioncited by 9
  26. 2020An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generationcited by 7
  27. 2020Machine Translation Pre-training for Data-to-Text Generation -- A Case Study in Czechcited by 5
  28. 2018Addressee and Response Selection for Multilingual Conversationcited by 4
  29. 2020Cross-lingual Transfer of Abstractive Summarizer to Less-resource Languagecited by 4
  30. 2020Pivot Through English: Reliably Answering Multilingual Questions without Document Retrievalcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 1; 2025: 2; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  2. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 117
  3. 2023ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessmentcited by 19
  4. 2024Bridging Cultural Nuances in Dialogue Agents through Cultural Value Surveyscited by 18
  5. 2023Unsupervised Layer-wise Score Aggregation for Textual OOD Detectioncited by 15
  6. 2023xPQA: Cross-Lingual Product Question Answering across 12 Languagescited by 10
  7. 2023Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generationcited by 4
  8. 2023Comparing Styles across Languages: A Cross-Cultural Exploration of Politenesscited by 2
  9. 2023Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systemscited by 1
  10. 2025RusCode: Russian Cultural Code Benchmark for Text-to-Image Generationcited by 1
  11. 2023Is Translation Helpful? An Empirical Analysis of Cross-Lingual Transfer in Low-Resource Dialog Generationcited by 0
  12. 2025Empowering Global Voices: A Data-Efficient, Phoneme-Tone Adaptive Approach to High-Fidelity Speech Synthesiscited by 0
1 more
  1. 2026CultureTalk-ID: A Multi-Task Dialogue Benchmark for Cultural Commonsense in Indonesian Local Languagescited by 0