mlsanityData to 14 Oct 2026

PILOT: A Pre-Trained Model-Based Continual Learning Toolbox

arXiv 2309.07117 ↗First posted 13 Sep 2023cs.LG cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

While traditional machine learning can effectively tackle a wide range of problems, it primarily operates within a closed-world setting, which presents limitations when dealing with streaming data. As a solution, incremental learning emerges to address real-world scenarios involving new data's arrival. Recently, pre-training has made significant advancements and garnered the attention of numerous researchers. The strong performance of these pre-trained models (PTMs) presents a promising avenue for developing continual learning algorithms that can effectively adapt to real-world scenarios. Consequently, exploring the utilization of PTMs in incremental learning has become essential. This paper introduces a pre-trained model-based continual learning toolbox known as PILOT. On the one hand, PILOT implements some state-of-the-art class-incremental learning algorithms based on pre-trained models, such as L2P, DualPrompt, and CODA-Prompt. On the other hand, PILOT also fits typical class-incremental learning algorithms (e.g., DER, FOSTER, and MEMO) within the context of pre-trained models to evaluate their effectiveness.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 13 Sep 2023.

Builds on

38 of 56 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  9. 2022Visual Prompt Tuningcited by 1,681
  10. 2019Natural Adversarial Examplescited by 1,408
  11. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  12. 2021Learning to Prompt for Continual Learningcited by 944
26 more
  1. 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
  2. 2021ImageNet-21K Pretraining for the Massescited by 613
  3. 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
  4. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 449
  5. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 425
  6. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  7. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  8. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  9. 2022FOSTER: Feature Boosting and Compression for Class-Incremental Learningcited by 229
  10. 2023Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Needcited by 184
  11. 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 180
  12. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  13. 2022A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learningcited by 154
  14. 2020Adaptive Aggregation Networks for Class-Incremental Learningcited by 149
  15. 2023A Unified Continual Learning Framework with General Parameter-Efficient Tuningcited by 125
  16. 2021Avalanche: an End-to-End Library for Continual Learningcited by 124
  17. 2021PyCIL: A Python Toolbox for Class-Incremental Learningcited by 83
  18. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  19. 2023RMM: Reinforced Memory Management for Class-Incremental Learningcited by 66
  20. 2020MgSvF: Multi-Grained Slow vs. Fast Framework for Few-Shot Class-Incremental Learningcited by 62
  21. 2022SparCL: Sparse Continual Learning on the Edgecited by 55
  22. 2020Memory Efficient Class-Incremental Learning for Image Classificationcited by 50
  23. 2022Exploring Example Influence in Continual Learningcited by 44
  24. 2021Co-Transport for Class-Incremental Learningcited by 38
  25. 2022Benchmarking Omni-Vision Representation through the Lens of Visual Realmscited by 38
  26. 2022A simple but strong baseline for online continual learning: Repeated Augmented Rehearsalcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 22; 2025: 18; 2026: 21.

64 papers.

Sort:

By area:

  1. 2024Continual Learning with Pre-Trained Models: A Surveycited by 154
  2. 2024Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learningcited by 117
  3. 2024Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learningcited by 53
  4. 2024Class-Incremental Learning with CLIP: Adaptive Representation Adjustment and Parameter Fusioncited by 47
  5. 2023Few-Shot Class-Incremental Learning via Training-Free Prototype Calibrationcited by 46
  6. 2023Weighted Ensemble Models Are Strong Continual Learnerscited by 42
  7. 2024MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learningcited by 36
  8. 2025Language Guided Concept Bottleneck Models for Interpretable Continual Learningcited by 28
  9. 2025CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learningcited by 28
  10. 2024A Practitioner's Guide to Continual Multimodal Pretrainingcited by 23
  11. 2024Generative Multi-modal Models are Good Class-Incremental Learnerscited by 22
  12. 2024Dual Consolidation for Pre-Trained Model-Based Domain-Incremental Learningcited by 16
38 more
  1. 2024Few-Shot Class Incremental Learning with Attention-Aware Self-Adaptive Promptcited by 15
  2. 2025Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Expertscited by 13
  3. 2025MCITlib: Multimodal Continual Instruction Tuning Library and Benchmarkcited by 13
  4. 2024Hyperparameters in Continual Learning: A Reality Checkcited by 12
  5. 2024F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learningcited by 12
  6. 2024Random Representations Outperform Online Continually Learned Representationscited by 11
  7. 2025Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learningcited by 11
  8. 2024Calibrating Higher-Order Statistics for Few-Shot Class-Incremental Learning with Pre-trained Vision Transformerscited by 9
  9. 2025ZeroFlow: Overcoming Catastrophic Forgetting is Easier than You Thinkcited by 9
  10. 2024LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learningcited by 8
  11. 2025Order-Robust Class Incremental Learning: Graph-Driven Dynamic Similarity Groupingcited by 7
  12. 2025Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learningcited by 3
  13. 2026C3Box: A CLIP-based Class-Incremental Learning Toolboxcited by 3
  14. 2026Scaling Continual Learning to 300+ Tasks with Bi-Level Routing Mixture-of-Expertscited by 3
  15. 2023Enhancing Plasticity for First Session Adaptation Continual Learningcited by 2
  16. 2024Semantic Prompting with Image-Token for Continual Learningcited by 2
  17. 2024TV100: A TV Series Dataset that Pre-Trained CLIP Has Not Seencited by 2
  18. 2024Sparse Orthogonal Parameters Tuning for Continual Learningcited by 2
  19. 2025MoTE: Mixture of Task-specific Experts for Pre-Trained ModelBased Class-incremental Learningcited by 2
  20. 2025The Lie of the Average: How Class Incremental Learning Evaluation Deceives You?cited by 2
  21. 2025Domain Generalizable Continual Learningcited by 2
  22. 2024IncSAR: A Dual Fusion Incremental Learning Framework for SAR Target Recognitioncited by 1
  23. 2024Enhancing Multimodal Large Language Models Complex Reason via Similarity Computationcited by 1
  24. 2024PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learningcited by 1
  25. 2025Knowledge Graph Enhanced Generative Multi-modal Models for Class-Incremental Learningcited by 1
  26. 2025Forward-Only Continual Learningcited by 1
  27. 2025More Than Memory Savings: Zeroth-Order Optimization Mitigates Forgetting in Continual Learningcited by 1
  28. 2026Temporal Imbalance of Positive and Negative Supervision in Class-Incremental Learningcited by 1
  29. 2026Can You Hear, Localize, and Segment Continually? An Exemplar-Free Continual Learning Benchmark for Audio-Visual Segmentationcited by 1
  30. 2026One Adapter for All: Towards Unified Representation in Step-Imbalanced Class-Incremental Learningcited by 1
  31. 2026Stable Routing for Mixture-of-Experts in Class-Incremental Learningcited by 1
  32. 2024Realistic Continual Learning Approach using Pre-trained Modelscited by 0
  33. 2024Dual Prototypes for Adaptive Pre-Trained Model in Class-Incremental Learningcited by 0
  34. 2025Unlocking [CLS] Features for Continual Post-Trainingcited by 0
  35. 2025From Offline to Online Memory-Free and Task-Free Continual Learning via Fine-Grained Hypergradientscited by 0
  36. 2025Adapt before Continual Learningcited by 0
  37. 2025OpenHAIV: A Framework Towards Practical Open-World Learningcited by 0
  38. 2026Dynamical Adapter Fusion: Constructing A Global Adapter for Pre-Trained Model-based Class-Incremental Learningcited by 0

50 most cited of 64 All 64 (JSON)