mlsanityData to 7 Oct 2026

Recommendation of data-free class-incremental learning algorithms by simulating future data

arXiv 2403.18132 ↗First posted 26 Mar 2024cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Class-incremental learning deals with sequential data streams composed of batches of classes. Various algorithms have been proposed to address the challenging case where samples from past classes cannot be stored. However, selecting an appropriate algorithm for a user-defined setting is an open problem, as the relative performance of these algorithms depends on the incremental settings. To solve this problem, we introduce an algorithm recommendation method that simulates the future data stream. Given an initial set of classes, it leverages generative models to simulate future classes from the same visual domain. We evaluate recent algorithms on the simulated stream and recommend the one which performs best in the user-defined incremental setting. We illustrate the effectiveness of our method on three large datasets using six algorithms and six incremental settings. Our method outperforms competitive baselines, and performance is close to that of an oracle choosing the best algorithm in each setting. This work contributes to facilitate the practical deployment of incremental learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 54 of its 72 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

54 of 72 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Generative Adversarial Networkscited by 5,603
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2016Learning without Forgettingcited by 2,465
38 more
  1. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2016Progressive Neural Networkscited by 1,332
  7. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  8. 2017Continual Learning with Deep Generative Replaycited by 1,152
  9. 2021Learning to Prompt for Continual Learningcited by 938
  10. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  11. 2018End-to-End Incremental Learningcited by 596
  12. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  13. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 369
  14. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 359
  15. 2020Semantic Drift Compensation for Class-Incremental Learningcited by 241
  16. 2016Large-Scale Image Retrieval with Attentive Deep Local Featurescited by 238
  17. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  18. 2019REMIND Your Neural Network to Prevent Catastrophic Forgettingcited by 197
  19. 2023StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learnerscited by 177
  20. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 171
  21. 2021Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learningcited by 130
  22. 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 120
  23. 2022Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learningcited by 118
  24. 2020A Comprehensive Study of Class Incremental Learning Algorithms for Visual Taskscited by 107
  25. 2022Constrained Few-shot Class-incremental Learningcited by 93
  26. 2019Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysiscited by 91
  27. 2023FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learningcited by 86
  28. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 83
  29. 2023Computationally Budgeted Continual Learning: What Does Matter?cited by 61
  30. 2023Continual Learning: Applications and the Road Forwardcited by 59
  31. 2022A Simple Baseline that Questions the Use of Pretrained-Models in Continual Learningcited by 54
  32. 2022Online Continual Learning for Embedded Devicescited by 48
  33. 2021Class-Incremental Learning with Generative Classifierscited by 39
  34. 2023Class-Incremental Learning using Diffusion Model for Distillation and Replaycited by 31
  35. 2022A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspectivecited by 27
  36. 2023How Efficient Are Today's Continual Learning Algorithms?cited by 24
  37. 2023TIAM -- A Metric for Evaluating Alignment in Text-to-Image Generationcited by 18
  38. 2021Balanced softmax cross-entropy for incremental learning with and without memorycited by 13

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.