mlsanityData to 7 Oct 2026

Prompt Learning to Mitigate Catastrophic Forgetting in Cross-lingual Transfer for Open-domain Dialogue Generation

arXiv 2305.07393 ↗First posted 12 May 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Dialogue systems for non-English languages have long been under-explored. In this paper, we take the first step to investigate few-shot cross-lingual transfer learning (FS-XLT) and multitask learning (MTL) in the context of open-domain dialogue generation for non-English languages with limited data. We observed catastrophic forgetting in both FS-XLT and MTL for all 6 languages in our preliminary experiments. To mitigate the issue, we propose a simple yet effective prompt learning approach that can preserve the multilinguality of multilingual pre-trained language model (mPLM) in FS-XLT and MTL by bridging the gap between pre-training and fine-tuning with Fixed-prompt LM Tuning and our hand-crafted prompts. Experimental results on all 6 languages in terms of both automatic and human evaluations demonstrate the effectiveness of our approach. Our code is available at https://github.com/JeremyLeiLiu/XLinguDial.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

42 of 51 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  9. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  11. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  12. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
30 more
  1. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  2. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  3. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  4. 2020Recipes for building an open-domain chatbotcited by 593
  5. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  6. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  7. 2019PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variablecited by 145
  8. 2015Multi-task Sequence to Sequence Learningcited by 94
  9. 2020Few-Shot Text Generation with Pattern-Exploiting Trainingcited by 87
  10. 2020PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learningcited by 79
  11. 2020A Large-Scale Chinese Short-Text Conversation Datasetcited by 77
  12. 2021Few-Shot Bot: Prompt-Based Learning for Dialogue Systemscited by 64
  13. 2019Cross-Lingual Natural Language Generation via Pre-Trainingcited by 63
  14. 2022Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generationcited by 62
  15. 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
  16. 2020A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matterscited by 45
  17. 2021MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairscited by 41
  18. 2021PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generationcited by 37
  19. 2020XPersona: Evaluating Multilingual Personalized Chatbotcited by 36
  20. 2021EVA: An Open-Domain Chinese Dialogue System with Large-Scale Generative Pre-Trainingcited by 34
  21. 2022EVA2.0: Investigating Open-Domain Chinese Dialogue Systems with Large-Scale Pre-Trainingcited by 31
  22. 2021ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generationcited by 23
  23. 2021Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systemscited by 19
  24. 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learningcited by 17
  25. 2018Multi-Task Learning for Sequence Tagging: An Empirical Studycited by 15
  26. 2022MDIA: A Benchmark for Multilingual Dialogue Generation in 46 Languagescited by 13
  27. 2021Smaaprat: DialoGPT for Natural Language Generation of Swedish Dialogue by Transfer Learningcited by 11
  28. 2022Building a Personalized Dialogue System with Prompt-Tuningcited by 9
  29. 2022Computational historical linguistics and language diversity in South Asiacited by 8
  30. 2021Empathetic BERT2BERT Conversational Model: Learning Arabic Language Generation with Little Datacited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 2.

4 papers.

  1. 2023TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Modelscited by 59
  2. 2024Soft Language Prompts for Language Transfercited by 5
  3. 2025MLLM-CTBench: A Benchmark for Continual Instruction Tuning with Reasoning Process Diagnosiscited by 2
  4. 2025Mixtures of SubExperts for Large Language Continual Learningcited by 0