# iTAML: An Incremental Task-Agnostic Meta-learning Approach

arXiv: https://arxiv.org/abs/2003.11652 · HTML: https://mlsanity.com/paper/2003.11652/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2020.

## Built on by (most cited first, 30 of 90)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks](https://mlsanity.com/paper/2203.17030/) (2022)
- [On Learning the Geodesic Path for Incremental Learning](https://mlsanity.com/paper/2104.08572/) (2021)
- [Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks](https://mlsanity.com/paper/2112.10017/) (2021)
- [A Theoretical Study on Solving Continual Learning](https://mlsanity.com/paper/2211.02633/) (2022)
- [RMM: Reinforced Memory Management for Class-Incremental Learning](https://mlsanity.com/paper/2301.05792/) (2023)
- [Computationally Budgeted Continual Learning: What Does Matter?](https://mlsanity.com/paper/2303.11165/) (2023)
- [Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks](https://mlsanity.com/paper/2105.12374/) (2021)
- [Recent Advances of Continual Learning in Computer Vision: An Overview](https://mlsanity.com/paper/2109.11369/) (2021)
- [Efficient Feature Transformations for Discriminative and Generative Continual Learning](https://mlsanity.com/paper/2103.13558/) (2021)
- [AttriCLIP: A Non-Incremental Learner for Incremental Knowledge Learning](https://mlsanity.com/paper/2305.11488/) (2023)
- [When Prompt-based Incremental Learning Does Not Meet Strong Pretraining](https://mlsanity.com/paper/2308.10445/) (2023)
- [UC-OWOD: Unknown-Classified Open World Object Detection](https://mlsanity.com/paper/2207.11455/) (2022)
- [Understanding Continual Learning Settings with Data Distribution Drift Analysis](https://mlsanity.com/paper/2104.01678/) (2021)
- [Energy-based Latent Aligner for Incremental Learning](https://mlsanity.com/paper/2203.14952/) (2022)
- [Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting](https://mlsanity.com/paper/2108.08165/) (2021)
- [Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning](https://mlsanity.com/paper/2202.10203/) (2022)
- [Learnability and Algorithm for Continual Learning](https://mlsanity.com/paper/2306.12646/) (2023)
- [A Multi-Head Model for Continual Learning via Out-of-Distribution Replay](https://mlsanity.com/paper/2208.09734/) (2022)
- [Optimizing Reusable Knowledge for Continual Learning via Metalearning](https://mlsanity.com/paper/2106.05390/) (2021)
- [Continual Object Detection: A review of definitions, strategies, and challenges](https://mlsanity.com/paper/2205.15445/) (2022)
- [Random Boxes Are Open-world Object Detectors](https://mlsanity.com/paper/2307.08249/) (2023)
- [Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy](https://mlsanity.com/paper/2312.10549/) (2023)
- [Open-World Continual Learning: Unifying Novelty Detection and Continual Learning](https://mlsanity.com/paper/2304.10038/) (2023)
- [Continual Learning Based on OOD Detection and Task Masking](https://mlsanity.com/paper/2203.09450/) (2022)

## Abstract (arXiv)

Humans can continuously learn new knowledge as their experience grows. In contrast, previous learning in deep neural networks can quickly fade out when they are trained on a new task. In this paper, we hypothesize this problem can be avoided by learning a set of generalized parameters, that are neither specific to old nor new tasks. In this pursuit, we introduce a novel meta-learning approach that seeks to maintain an equilibrium between all the encountered tasks. This is ensured by a new meta-update rule which avoids catastrophic forgetting. In comparison to previous meta-learning techniques, our approach is task-agnostic. When presented with a continuum of data, our model automatically identifies the task and quickly adapts to it with just a single update. We perform extensive experiments on five datasets in a class-incremental setting, leading to significant improvements over the state of the art methods (e.g., a 21.3% boost on CIFAR100 with 10 incremental tasks). Specifically, on large-scale datasets that generally prove difficult cases for incremental learning, our approach delivers absolute gains as high as 19.1% and 7.4% on ImageNet and MS-Celeb datasets, respectively.