mlsanityData to 14 Oct 2026

Towards Robustness and Diversity: Continual Learning in Dialog Generation with Text-Mixup and Batch Nuclear-Norm Maximization

arXiv 2403.10894 ↗First posted 16 Mar 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

In our dynamic world where data arrives in a continuous stream, continual learning enables us to incrementally add new tasks/domains without the need to retrain from scratch. A major challenge in continual learning of language model is catastrophic forgetting, the tendency of models to forget knowledge from previously trained tasks/domains when training on new ones. This paper studies dialog generation under the continual learning setting. We propose a novel method that 1) uses textitText-Mixup as data augmentation to avoid model overfitting on replay memory and 2) leverages Batch-Nuclear Norm Maximization (BNNM) to alleviate the problem of mode collapse. Experiments on a 37-domain task-oriented dialog dataset and DailyDialog (a 10-domain chitchat dataset) demonstrate that our proposed approach outperforms the state-of-the-art in continual learning.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 16 Mar 2024.

Builds on

31 of 47 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  5. 2016Learning without Forgettingcited by 2,496
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  7. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  8. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  9. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  10. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  11. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  12. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
19 more
  1. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  2. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
  3. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
  4. 2021A Survey of Data Augmentation Approaches for NLPcited by 409
  5. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 364
  6. 2020How Does Mixup Help With Robustness and Generalization?cited by 193
  7. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
  8. 2020Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situationscited by 133
  9. 2019A continual learning survey: Defying forgetting in classification taskscited by 118
  10. 2020Convolutional Networks with Dense Connectivitycited by 98
  11. 2021Representational Continuity for Unsupervised Continual Learningcited by 95
  12. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  13. 2020Continual Learning in Task-Oriented Dialogue Systemscited by 83
  14. 2022Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignmentcited by 54
  15. 2020Continual Learning for Natural Language Generation in Task-oriented Dialog Systemscited by 47
  16. 2020CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directionscited by 26
  17. 2018On Catastrophic Forgetting and Mode Collapse in Generative Adversarial Networkscited by 17
  18. 2021CAM-loss: Towards Learning Spatially Discriminative Feature Representationscited by 13
  19. 2024Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalizationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Emotional Support with LLM-based Empathetic Dialogue Generationcited by 4
  2. 2025Multi-Intent Spoken Language Understanding: Methods, Trends, and Challengescited by 0
  3. 2026Impact of enriched meaning representations for language generation in dialogue tasks: A comprehensive exploration of the relevance of tasks, corpora and metricscited by 0