mlsanityData to 14 Oct 2026

One Size Fits All for Semantic Shifts: Adaptive Prompt Tuning for Continual Learning

arXiv 2311.12048 ↗First posted 18 Nov 2023cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

In real-world continual learning (CL) scenarios, tasks often exhibit intricate and unpredictable semantic shifts, posing challenges for fixed prompt management strategies which are tailored to only handle semantic shifts of uniform degree (i.e., uniformly mild or uniformly abrupt). To address this limitation, we propose an adaptive prompting approach that effectively accommodates semantic shifts of varying degree where mild and abrupt shifts are mixed. AdaPromptCL employs the assign-and-refine semantic grouping mechanism that dynamically manages prompt groups in accordance with the semantic similarity between tasks, enhancing the quality of grouping through continuous refinement. Our experiment results demonstrate that AdaPromptCL outperforms existing prompting methods by up to 21.3%, especially in the benchmark datasets with diverse semantic shifts between tasks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

29 of 49 references.

References · 29

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  4. 2022Visual Prompt Tuningcited by 1,681
  5. 2018Experience Replay for Continual Learningcited by 1,095
  6. 2021Learning to Prompt for Continual Learningcited by 944
  7. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  8. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  9. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
  10. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  11. 2018Taskonomy: Disentangling Task Transfer Learningcited by 634
  12. 2019Gradient based sample selection for online continual learningcited by 571
17 more
  1. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
  2. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  3. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  4. 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 375
  5. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  6. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  7. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  8. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 245
  9. 2021Efficiently Identifying Task Groupings for Multi-Task Learningcited by 222
  10. 2020A Neural Dirichlet Process Mixture Model for Task-Free Continual Learningcited by 134
  11. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 125
  12. 2020Learning to Branch for Multi-Task Learningcited by 115
  13. 2020Understanding and Improving Information Transfer in Multi-Task Learningcited by 114
  14. 2020A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learningcited by 87
  15. 2021Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inferencecited by 67
  16. 2022AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learningcited by 31
  17. 2021Is Class-Incremental Enough for Continual Learning?cited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 4.

8 papers.

  1. 2024Hyperparameters in Continual Learning: A Reality Checkcited by 12
  2. 2025When Continue Learning Meets Multimodal Large Language Model: A Surveycited by 9
  3. 2024LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learningcited by 8
  4. 2025Revisit Visual Prompt Tuning: The Expressiveness of Prompt Expertscited by 5
  5. 2026Beyond Perfect APIs: A Comprehensive Evaluation of LLM Agents Under Real-World API Complexitycited by 5
  6. 2026Continual Fine-Tuning with Provably Accurate and Parameter-Free Task Retrievalcited by 0
  7. 2026HiVe: Beyond Static Prompts for Multitask Learning via Hierarchy-based Vertical Mixture-of-Expertscited by 0
  8. 2026Ready2Blend: From Natural-Language Instructions to Composable Alignment Promptscited by 0