mlsanityData to 14 Oct 2026

Incremental Learning of Structured Memory via Closed-Loop Transcription

arXiv 2202.05411 ↗First posted 11 Feb 2022cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

This work proposes a minimal computational model for learning structured memories of multiple object classes in an incremental setting. Our approach is based on establishing a closed-loop transcription between the classes and a corresponding set of subspaces, known as a linear discriminative representation, in a low-dimensional feature space. Our method is simpler than existing approaches for incremental learning, and more efficient in terms of model size, storage, and computation: it requires only a single, fixed-capacity autoencoding network with a feature space that is used for both discriminative and generative purposes. Network parameters are optimized simultaneously without architectural manipulations, by solving a constrained minimax game between the encoding and decoding maps over a single rate reduction-based objective. Experimental results show that our method can effectively alleviate catastrophic forgetting, achieving significantly better performance than prior work of generative replay on MNIST, CIFAR-10, and ImageNet-50, despite requiring fewer resources. Source code can be found at https://github.com/tsb0601/i-CTRL

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 28 of its 49 references that are in the map. First posted on arXiv 11 Feb 2022.

Builds on

28 of 49 references.

References · 28

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2016Learning without Forgettingcited by 2,496
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  7. 2017Improved Training of Wasserstein GANscited by 2,235
  8. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  9. 2016Progressive Neural Networkscited by 1,361
  10. 2017Continual Learning with Deep Generative Replaycited by 1,158
  11. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  12. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
16 more
  1. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  2. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  3. 2018Progress & Compress: A scalable framework for continual learningcited by 520
  4. 2021DER: Dynamically Expandable Representation for Class Incremental Learningcited by 355
  5. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  6. 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 182
  7. 2017FearNet: Brain-Inspired Model for Incremental Learningcited by 177
  8. 2020Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reductioncited by 142
  9. 2022Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning Systemcited by 112
  10. 2019Continual Learning via Neural Pruningcited by 101
  11. 2021ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reductioncited by 84
  12. 2021EEC: Learning to Encode and Regenerate Images for Continual Learningcited by 40
  13. 2020Incremental Learning via Rate Reductioncited by 31
  14. 2013Robust subspace clusteringcited by 24
  15. 2021Closed-Loop Data Transcription to an LDR via Minimaxing Rate Reductioncited by 17
  16. 2019Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networkscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 2; 2025: 4; 2026: 1.

17 papers.

Sort:

By area:

  1. 2022On the Principles of Parsimony and Self-Consistency for the Emergence of Intelligencecited by 50
  2. 2023Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Modelscited by 22
  3. 2023EMP-SSL: Towards Self-Supervised Learning in One Training Epochcited by 19
  4. 2024Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embeddingcited by 10
  5. 2023Exploring Compositional Visual Generation with Latent Classifier Guidancecited by 9
  6. 2025Federated Class-Incremental Learning: A Hybrid Approach Using Latent Exemplars and Data-Free Techniques to Address Local and Global Forgettingcited by 9
  7. 2022Unsupervised Learning of Structured Representations via Closed-Loop Transcriptioncited by 7
  8. 2022Sustaining Fairness via Incremental Learningcited by 5
  9. 2022Pursuit of a Discriminative Representation for Multiple Subspaces via Sequential Gamescited by 4
  10. 2023Closed-Loop Transcription via Convolutional Sparse Codingcited by 4
  11. 2025Autoencoder-Based Hybrid Replay for Class-Incremental Learningcited by 4
  12. 2024Ctrl123: Consistent Novel View Synthesis via Closed-Loop Transcriptioncited by 2
5 more
  1. 2025Frugal Incremental Generative Modeling using Variational Autoencoderscited by 1
  2. 2023On Interpretable Approaches to Cluster, Classify and Represent Multi-Subspace Data via Minimum Lossy Coding Length based on Rate-Distortion Theorycited by 0
  3. 2023Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learningcited by 0
  4. 2025Deep Generative Continual Learning using Functional LoRA: FunLoRAcited by 0
  5. 2026On the Limits of Maximal Coding Rate Reduction for Out-of-Distribution Generalisationcited by 0