# TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation

arXiv: https://arxiv.org/abs/2408.09857 · HTML: https://mlsanity.com/paper/2408.09857/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 23 of 23)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [Movement Pruning: Adaptive Sparsity by Fine-Tuning](https://mlsanity.com/paper/2005.07683/) (2020)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Task-Specific Skill Localization in Fine-tuned Language Models](https://mlsanity.com/paper/2302.06600/) (2023)
- [Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning](https://mlsanity.com/paper/2112.02706/) (2021)
- [Continual Learning in Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2012.15504/) (2020)
- [Continual Learning of Natural Language Processing Tasks: A Survey](https://mlsanity.com/paper/2211.12701/) (2022)
- [PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance](https://mlsanity.com/paper/2206.12562/) (2022)
- [Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization](https://mlsanity.com/paper/2105.12002/) (2021)
- [Weighted Ensemble Models Are Strong Continual Learners](https://mlsanity.com/paper/2312.08977/) (2023)
- [Parameter-Level Soft-Masking for Continual Learning](https://mlsanity.com/paper/2306.14775/) (2023)
- [Continual Learning for Task-oriented Dialogue System with Iterative Network Pruning, Expanding and Masking](https://mlsanity.com/paper/2107.08173/) (2021)
- [Continual Dialogue State Tracking via Example-Guided Question Answering](https://mlsanity.com/paper/2305.13721/) (2023)

## Built on by (most cited first, 18 of 18)

- [HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Models](https://mlsanity.com/paper/2506.03922/) (2025)
- [Recurrent Knowledge Identification and Fusion for Language Model Continual Learning](https://mlsanity.com/paper/2502.17510/) (2025)
- [Gated Integration of Low-Rank Adaptation for Continual Learning of Large Language Models](https://mlsanity.com/paper/2505.15424/) (2025)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models](https://mlsanity.com/paper/2503.16843/) (2025)
- [FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning](https://mlsanity.com/paper/2601.03938/) (2026)
- [Locate-then-Merge: Neuron-Level Parameter Fusion for Mitigating Catastrophic Forgetting in Multimodal LLMs](https://mlsanity.com/paper/2505.16703/) (2025)
- [GeoEdit: Geometric Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2502.19953/) (2025)
- [PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs](https://mlsanity.com/paper/2506.17901/) (2025)
- [AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning](https://mlsanity.com/paper/2509.17348/) (2025)
- [ADEPT: Continual Pretraining via Adaptive Expansion and Dynamic Decoupled Tuning](https://mlsanity.com/paper/2510.10071/) (2025)
- [DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Search](https://mlsanity.com/paper/2508.20353/) (2025)
- [LoSiA: Efficient High-Rank Fine-Tuning via Subnet Localization and Optimization](https://mlsanity.com/paper/2507.04487/) (2025)
- [Enhancing the Preference Extractor in Multi-turn Dialogues: From Annotating Disasters to Accurate Preference Extraction](https://mlsanity.com/paper/2508.01739/) (2025)
- [Mixtures of SubExperts for Large Language Continual Learning](https://mlsanity.com/paper/2511.06237/) (2025)
- [CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning](https://mlsanity.com/paper/2605.05732/) (2026)
- [Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Models](https://mlsanity.com/paper/2605.28828/) (2026)
- [Latent-LoRA: Compact Latent-Space Adapters with Gradient-Free Routing for Continual Learning](https://mlsanity.com/paper/2607.23837/) (2026)

## Abstract (arXiv)

A practical dialogue system requires the capacity for ongoing skill acquisition and adaptability to new tasks while preserving prior knowledge. However, current methods for Continual Dialogue State Tracking (DST), a crucial function of dialogue systems, struggle with the catastrophic forgetting issue and knowledge transfer between tasks. We present TaSL, a novel framework for task skill localization and consolidation that enables effective knowledge transfer without relying on memory replay. TaSL uses a novel group-wise technique to pinpoint task-specific and task-shared areas. Additionally, a fine-grained skill consolidation strategy protects task-specific knowledge from being forgotten while updating shared knowledge for bi-directional knowledge transfer. As a result, TaSL strikes a balance between preserving previous knowledge and excelling at new tasks. Comprehensive experiments on various backbones highlight the significant performance improvements of TaSL over existing state-of-the-art methods. The source code is provided for reproducibility.