mlsanityData to 7 Oct 2026

REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning

arXiv 2406.04772 ↗First posted 7 Jun 2024cs.LG cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent rehearsal-free continual learning (CL) methods guided by prompts achieve strong performance on vision tasks with non-stationary data but remain resource-intensive, hindering real-world edge deployment. We introduce resource-efficient prompting (REP), which improves the computational and memory efficiency of prompt-based rehearsal-free continual learning methods while minimizing accuracy trade-offs. Our approach employs swift prompt selection to refine input data using a carefully provisioned model and introduces adaptive token merging (AToM) and adaptive layer dropping (ALD) for efficient prompt updates. AToM and ALD selectively skip data and model layers while preserving task-specific features during the learning of new tasks. Extensive experiments on multiple image classification datasets demonstrate REP's superior resource efficiency over state-of-the-art rehearsal-free CL methods.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 36 of its 57 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

36 of 57 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  8. 2022Token Merging: Your ViT But Fastercited by 995
  9. 2016Deep Networks with Stochastic Depthcited by 973
  10. 2021Learning to Prompt for Continual Learningcited by 938
  11. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  12. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
24 more
  1. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  2. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 635
  3. 2018End-to-End Incremental Learningcited by 596
  4. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  5. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 446
  6. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  7. 2022S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learningcited by 246
  8. 2016Using Deep Learning for Image-Based Plant Disease Detectioncited by 216
  9. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 183
  10. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 178
  11. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 162
  12. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 152
  13. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 124
  14. 2023Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimalitycited by 121
  15. 2020Accelerating Training of Transformer-Based Language Models with Progressive Layer Droppingcited by 93
  16. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  17. 2022Online Continual Learning for Embedded Devicescited by 48
  18. 2019E2-Train: Training State-of-the-art CNNs with Over 80% Energy Savingscited by 36
  19. 2024Convolutional Prompting meets Language Models for Continual Learningcited by 32
  20. 2024OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learningcited by 20
  21. 2021Exploring System Performance of Continual Learning for Mobile and Embedded Sensing Applicationscited by 17
  22. 2024F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learningcited by 12
  23. 2023Cost-effective On-device Continual Learning over Memory Hierarchy with Mirocited by 10
  24. 2018Random Projection in Deep Neural Networkscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024PECTP: Parameter-Efficient Cross-Task Prompts for Incremental Vision Transformercited by 8
  2. 2026Learning Probabilistic Prompt for Continual Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edge
  2. 2024Mixture of Experts Meets Prompt-Based Continual Learning
  3. 2025One-Prompt Strikes Back: Sparse Mixture of Experts for Prompt-based Continual Learning
  4. 2024Caption, Create, Continue: Continual Learning with Pre-trained Generative Vision-Language Models
  5. 2024Dual-Phase Continual Learning: Supervised Adaptation Meets Unsupervised Retention
  6. 2025Memory-Free Continual Learning with Null Space Adaptation for Zero-Shot Vision-Language Models
  7. 2026Decomposing and Composing: Towards Efficient Vision-Language Continual Learning via Rank-1 Expert Pool in a Single LoRA
  8. 2025IAP: Improving Continual Learning of Vision-Language Models via Instance-Aware Prompting