Frugal Incremental Generative Modeling using Variational Autoencoders
Not on the map. Why?
Abstract (arXiv)
Continual or incremental learning holds tremendous potential in deep learning with different challenges including catastrophic forgetting. The advent of powerful foundation and generative models has propelled this paradigm even further, making it one of the most viable solution to train these models. However, one of the persisting issues lies in the increasing volume of data particularly with replay-based methods. This growth introduces challenges with scalability since continuously expanding data becomes increasingly demanding as the number of tasks grows. In this paper, we attenuate this issue by devising a novel replay-free incremental learning model based on Variational Autoencoders (VAEs). The main contribution of this work includes (i) a novel incremental generative modelling, built upon a well designed multi-modal latent space, and also (ii) an orthogonality criterion that mitigates catastrophic forgetting of the learned VAEs. The proposed method considers two variants of these VAEs: static and dynamic with no (or at most a controlled) growth in the number of parameters. Extensive experiments show that our method is (at least) an order of magnitude more ``memory-frugal'' compared to the closely related works while achieving SOTA accuracy scores.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 56 of its 76 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
56 of 76 references.
References · 56
- 2017Attention Is All You Needcited by 60,361
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Generative Adversarial Networkscited by 5,627
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
38 more
- 2022Visual Prompt Tuningcited by 1,681
- 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
- 2016Progressive Neural Networkscited by 1,361
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
- 2017Continual Learning with Deep Generative Replaycited by 1,158
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
- 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 742
- 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
- 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 681
- 2022DualPrompt: Complementary Prompting for Rehearsal-free Continual Learningcited by 642
- 2019Gradient based sample selection for online continual learningcited by 571
- 2022CODA-Prompt: COntinual Decomposed Attention-based Prompting for Rehearsal-Free Continual Learningcited by 450
- 2021Gradient Projection Memory for Continual Learningcited by 348
- 2022Understanding Diffusion Models: A Unified Perspectivecited by 340
- 2017Variational Continual Learningcited by 339
- 2017Overcoming Catastrophic Forgetting by Incremental Moment Matchingcited by 243
- 2018Selective Experience Replay for Lifelong Learningcited by 241
- 2016Unrolled Generative Adversarial Networkscited by 208
- 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
- 2023SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Modelcited by 180
- 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
- 2020Using Hindsight to Anchor Past Knowledge in Continual Learningcited by 161
- 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
- 2022FeTrIL: Feature Translation for Exemplar-Free Class-Incremental Learningcited by 121
- 2023Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Modelscited by 112
- 2020Generative Feature Replay For Class-Incremental Learningcited by 109
- 2018Generative replay with feedback connections as a general strategy for continual learningcited by 104
- 2021PyCIL: A Python Toolbox for Class-Incremental Learningcited by 83
- 2019Lifelong GAN: Continual Learning for Conditional Image Generationcited by 78
- 2024Consistent Prompting for Rehearsal-Free Continual Learningcited by 54
- 2023AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learningcited by 46
- 2022CLIP Itself is a Strong Fine-tuner: Achieving 85.7% and 88.0% Top-1 Accuracy with ViT-B and ViT-L on ImageNetcited by 35
- 2024Visual Prompt Tuning in Null Space for Continual Learningcited by 31
- 2024Semantic Residual Prompts for Continual Learningcited by 19
- 2025One-for-More: Continual Diffusion Model for Anomaly Detectioncited by 19
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.