mlsanityData to 7 Oct 2026

Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking

arXiv 2107.08173 ↗First posted 17 Jul 2021cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

This ability to learn consecutive tasks without forgetting how to perform previously trained problems is essential for developing an online dialogue system. This paper proposes an effective continual learning for the task-oriented dialogue system with iterative network pruning, expanding and masking (TPEM), which preserves performance on previously encountered tasks while accelerating learning progress on subsequent tasks. Specifically, TPEM (i) leverages network pruning to keep the knowledge for old tasks, (ii) adopts network expanding to create free weights for new tasks, and (iii) introduces task-specific network masking to alleviate the negative impact of fixed weights of old tasks on new tasks. We conduct extensive experiments on seven different tasks from three benchmark datasets and show empirically that TPEM leads to significantly improved results over the strong competitors. For reproducibility, we submit the code and data at: https://github.com/siat-nlp/TPEM

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 15 of its 26 references that are in the map. First posted on arXiv 17 Jul 2021.

Builds on

15 of 26 references.

References · 15

  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  2. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  3. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  4. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  5. 2019Compacting, Picking and Growing for Unforgetting Continual Learningcited by 207
  6. 2019Uncertainty-based Continual Learning with Adaptive Regularizationcited by 144
  7. 2018Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systemscited by 70
  8. 2019Global-to-local Memory Pointer Networks for Task-Oriented Dialoguecited by 47
  9. 2020Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialogcited by 37
  10. 2020One Person, One Model, One World: Learning Continual User Representation without Forgettingcited by 30
  11. 2019Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systemscited by 25
  12. 2016Conditional Generation and Snapshot Learning in Neural Dialogue Systemscited by 17
3 more
  1. 2017A Copy-Augmented Sequence-to-Sequence Architecture Gives Good Performance on Task-Oriented Dialoguecited by 14
  2. 2018Multi-level Memory for Task Oriented Dialogscited by 14
  3. 2021Iterative Network Pruning with Uncertainty Regularization for Lifelong Sentiment Classificationcited by 7

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3; 2024: 9; 2025: 5.

22 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2022Continual Learning of Natural Language Processing Tasks: A Surveycited by 83
  3. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 60
  4. 2024Recent Advances of Foundation Language Models-based Continual Learning: A Surveycited by 46
  5. 2022Continual Prompt Tuning for Dialog State Trackingcited by 44
  6. 2025SEFE: Superficial and Essential Forgetting Eliminator for Multimodal Continual Instruction Tuningcited by 32
  7. 2024TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidationcited by 18
  8. 2024Continual Learning Using a Kernel-Based Method Over Foundation Modelscited by 13
  9. 2022Dynamic Dialogue Policy for Continual Reinforcement Learningcited by 12
  10. 2025Continual Learning for Generative AI: From LLMs to MLLMs and Beyondcited by 9
  11. 2022Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialoguecited by 8
  12. 2024Continual Dialogue State Tracking via Reason-of-Select Distillationcited by 8
10 more
  1. 2025Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgettingcited by 8
  2. 2024In-context Continual Learning Assisted by an External Continual Learnercited by 7
  3. 2025AnaCP: Toward Upper-Bound Continual Learning via Analytic Contrastive Projectioncited by 7
  4. 2024Continual Learning Using Only Large Language Model Promptingcited by 5
  5. 2023Visually Grounded Continual Language Learning with Selective Specializationcited by 4
  6. 2025Self-Evolving LLMs via Continual Instruction Tuningcited by 4
  7. 2023Prompt Pool based Class-Incremental Continual Learning for Dialog State Trackingcited by 2
  8. 2024Gradual Learning: Optimizing Fine-Tuning with Partially Mastered Knowledge in Large Language Modelscited by 2
  9. 2022Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?cited by 1
  10. 2024Overcoming Growth-Induced Forgetting in Task-Agnostic Continual Learningcited by 1