mlsanityData to 7 Oct 2026

Meta-Consolidation for Continual Learning

arXiv 2010.00352 ↗First posted 1 Oct 2020cs.CV cs.AIcited by 40

Not on the map. Why?

Abstract (arXiv)

The ability to continuously learn and adapt itself to new tasks, without losing grasp of already acquired knowledge is a hallmark of biological learning systems, which current deep learning systems fall short of. In this work, we present a novel methodology for continual learning called MERLIN: Meta-Consolidation for Continual Learning. We assume that weights of a neural network boldsymbol psi, for solving task boldsymbol t, come from a meta-distribution p(boldsymbolpsi|t). This meta-distribution is learned and consolidated incrementally. We operate in the challenging online continual learning setting, where a data point is seen by the model only once. Our experiments with continual learning benchmarks of MNIST, CIFAR-10, CIFAR-100 and Mini-ImageNet datasets show consistent improvement over five baselines, including a recent state-of-the-art, corroborating the promise of MERLIN.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 10; 2023: 4; 2024: 4; 2025: 3; 2026: 1.

40 papers.

Sort:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021Towards Open World Object Detectioncited by 297
  3. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  4. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  5. 2021Posterior Meta-Replay for Continual Learningcited by 45
  6. 2022UC-OWOD: Unknown-Classified Open World Object Detectioncited by 40
  7. 2022Novel Class Discovery without Forgettingcited by 39
  8. 2022Energy-based Latent Aligner for Incremental Learningcited by 37
  9. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  10. 2021Continual Semi-Supervised Learning through Contrastive Interpolation Consistencycited by 25
  11. 2021Optimizing Reusable Knowledge for Continual Learning via Metalearningcited by 24
  12. 2021Formalizing the Generalization-Forgetting Trade-off in Continual Learningcited by 24
28 more
  1. 2023Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomycited by 24
  2. 2021Prototype-Guided Memory Replay for Continual Learningcited by 19
  3. 2021Objects in Semantic Topologycited by 18
  4. 2021CoMPS: Continual Meta Policy Searchcited by 15
  5. 2022Class-Incremental Learning with Cross-Space Clustering and Controlled Transfercited by 15
  6. 2022Lifelong Learning Metricscited by 13
  7. 2024Learning Useful Representations of Recurrent Neural Network Weight Matricescited by 13
  8. 2021CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networkscited by 12
  9. 2022Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuningcited by 12
  10. 2023When Meta-Learning Meets Online and Continual Learning: A Surveycited by 12
  11. 2021Hypernetworks for Continual Semi-Supervised Learningcited by 11
  12. 2022Learn the Time to Learn: Replay Scheduling in Continual Learningcited by 11
  13. 2025Parameter-Efficient Continual Fine-Tuning: A Surveycited by 11
  14. 2022Spacing Loss for Discovering Novel Categoriescited by 10
  15. 2022Hierarchically Structured Task-Agnostic Continual Learningcited by 9
  16. 2024Dynamic Transformer Architecture for Continual Learning of Multimodal Taskscited by 8
  17. 2021Overcoming Catastrophic Forgetting with Gaussian Mixture Replaycited by 7
  18. 2021Mixture-of-Variational-Experts for Continual Learningcited by 7
  19. 2024Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applicationscited by 7
  20. 2021Continual Learning with Fully Probabilistic Modelscited by 6
  21. 2023New Insights on Relieving Task-Recency Bias for Online Class Incremental Learningcited by 4
  22. 2021Meta-Learned Attribute Self-Gating for Continual Generalized Zero-Shot Learningcited by 2
  23. 2024Recall-Oriented Continual Learning with Generative Adversarial Meta-Modelcited by 1
  24. 2025The Bayesian Approach to Continual Learning: An Overviewcited by 1
  25. 2021Knowledge Consolidation based Class Incremental Online Learning with Limited Datacited by 0
  26. 2021Graceful Degradation and Related Fieldscited by 0
  27. 2025Task-conditioned Ensemble of Expert Models for Continuous Learningcited by 0
  28. 2026Fast and Slow Variational Continual Learningcited by 0