# Continual Learning with Adaptive Weights (CLAW)

arXiv: https://arxiv.org/abs/1911.09514 · HTML: https://mlsanity.com/paper/1911.09514/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 30 of 38)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Training Networks in Null Space of Feature Covariance for Continual Learning](https://mlsanity.com/paper/2103.07113/) (2021)
- [Continual World: A Robotic Benchmark For Continual Reinforcement Learning](https://mlsanity.com/paper/2105.10919/) (2021)
- [GCR: Gradient Coreset Based Replay Buffer Selection For Continual Learning](https://mlsanity.com/paper/2111.11210/) (2021)
- [Wild-Time: A Benchmark of in-the-Wild Distribution Shift over Time](https://mlsanity.com/paper/2211.14238/) (2022)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Posterior Meta-Replay for Continual Learning](https://mlsanity.com/paper/2103.01133/) (2021)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [Prototype-Guided Memory Replay for Continual Learning](https://mlsanity.com/paper/2108.12641/) (2021)
- [Human-inspired Perspectives: A Survey on AI Long-term Memory](https://mlsanity.com/paper/2411.00489/) (2024)
- [Learn the Time to Learn: Replay Scheduling in Continual Learning](https://mlsanity.com/paper/2209.08660/) (2022)
- [Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18999/) (2024)
- [Learning to Learn: How to Continuously Teach Humans and Machines](https://mlsanity.com/paper/2211.15470/) (2022)
- [Statistical mechanics of continual learning: variational principle and mean-field potential](https://mlsanity.com/paper/2212.02846/) (2022)
- [HyperTime: Hyperparameter Optimization for Combating Temporal Distribution Shifts](https://mlsanity.com/paper/2305.18421/) (2023)
- [Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications](https://mlsanity.com/paper/2410.09218/) (2024)
- [On Sequential Bayesian Inference for Continual Learning](https://mlsanity.com/paper/2301.01828/) (2023)
- [TAG: Task-based Accumulated Gradients for Lifelong learning](https://mlsanity.com/paper/2105.05155/) (2021)
- [Continual Learning of Unsupervised Monocular Depth from Videos](https://mlsanity.com/paper/2311.02393/) (2023)
- [Learning to Learn for Few-shot Continual Active Learning](https://mlsanity.com/paper/2311.03732/) (2023)
- [Tuned Compositional Feature Replays for Efficient Stream Learning](https://mlsanity.com/paper/2104.02206/) (2021)
- [Efficient and robust multi-task learning in the brain with modular latent primitives](https://mlsanity.com/paper/2105.14108/) (2021)
- [AdaptCL: Adaptive Continual Learning for Tackling Heterogeneity in Sequential Datasets](https://mlsanity.com/paper/2207.11005/) (2022)
- [Center Loss Regularization for Continual Learning](https://mlsanity.com/paper/2110.11314/) (2021)
- [Model Assembly Learning with Heterogeneous Layer Weight Merging](https://mlsanity.com/paper/2503.21657/) (2025)
- [Optimal Empirical Risk Minimization under Temporal Distribution Shifts](https://mlsanity.com/paper/2507.13287/) (2025)
- [Smaller Is Better: An Analysis of Instance Quantity/Quality Trade-off in Rehearsal-based Continual Learning](https://mlsanity.com/paper/2105.14106/) (2021)
- [EANet: Expert Attention Network for Online Trajectory Prediction](https://mlsanity.com/paper/2309.05683/) (2023)

## Abstract (arXiv)

Approaches to continual learning aim to successfully learn a set of related tasks that arrive in an online manner. Recently, several frameworks have been developed which enable deep learning to be deployed in this learning scenario. A key modelling decision is to what extent the architecture should be shared across tasks. On the one hand, separately modelling each task avoids catastrophic forgetting but it does not support transfer learning and leads to large models. On the other hand, rigidly specifying a shared component and a task-specific part enables task transfer and limits the model size, but it is vulnerable to catastrophic forgetting and restricts the form of task-transfer that can occur. Ideally, the network should adaptively identify which parts of the network to share in a data driven way. Here we introduce such an approach called Continual Learning with Adaptive Weights (CLAW), which is based on probabilistic modelling and variational inference. Experiments show that CLAW achieves state-of-the-art performance on six benchmarks in terms of overall continual learning performance, as measured by classification accuracy, and in terms of addressing catastrophic forgetting.