# Learning to Forget for Meta-Learning

arXiv: https://arxiv.org/abs/1906.05895 · HTML: https://mlsanity.com/paper/1906.05895/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2019.

## Built on by (most cited first, 30 of 35)

- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning](https://mlsanity.com/paper/2307.09218/) (2023)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning](https://mlsanity.com/paper/2307.16424/) (2023)
- [Sketch3T: Test-Time Training for Zero-Shot SBIR](https://mlsanity.com/paper/2203.14691/) (2022)
- [MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition](https://mlsanity.com/paper/2104.01876/) (2021)
- [Meta-learning approaches for few-shot learning: A survey of recent advances](https://mlsanity.com/paper/2303.07502/) (2023)
- [Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learning](https://mlsanity.com/paper/2110.14202/) (2021)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [Reference Twice: A Simple and Unified Baseline for Few-Shot Instance Segmentation](https://mlsanity.com/paper/2301.01156/) (2023)
- [Meta-Learning with a Geometry-Adaptive Preconditioner](https://mlsanity.com/paper/2304.01552/) (2023)
- [Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks](https://mlsanity.com/paper/2107.02378/) (2021)
- [Multi-level Metric Learning for Few-shot Image Recognition](https://mlsanity.com/paper/2103.11383/) (2021)
- [Grad2Task: Improved Few-shot Text Classification Using Gradients for Task Representation](https://mlsanity.com/paper/2201.11576/) (2022)
- [Information Symmetry Matters: A Modal-Alternating Propagation Network for Few-Shot Learning](https://mlsanity.com/paper/2109.01295/) (2021)
- [Self-Taught Cross-Domain Few-Shot Learning with Weakly Supervised Object Localization and Task-Decomposition](https://mlsanity.com/paper/2109.01302/) (2021)
- [Contextual Gradient Scaling for Few-Shot Learning](https://mlsanity.com/paper/2110.10353/) (2021)
- [Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks](https://mlsanity.com/paper/2205.15619/) (2022)
- [Learning with Constraint Learning: New Perspective, Solution Strategy and Various Applications](https://mlsanity.com/paper/2307.15257/) (2023)
- [Curvature Learning for Generalization of Hyperbolic Neural Networks](https://mlsanity.com/paper/2508.17232/) (2025)
- [Meta-learning of Pooling Layers for Character Recognition](https://mlsanity.com/paper/2103.09528/) (2021)
- [Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis](https://mlsanity.com/paper/2509.23915/) (2025)
- [Hierarchical Representation based Query-Specific Prototypical Network for Few-Shot Image Classification](https://mlsanity.com/paper/2103.11384/) (2021)
- [Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts using Meta-Learning Hypernetworks](https://mlsanity.com/paper/2307.06771/) (2023)
- [Writer adaptation for offline text recognition: An exploration of neural network-based methods](https://mlsanity.com/paper/2307.15071/) (2023)
- [Hyperbolic Dual Feature Augmentation for Open-Environment](https://mlsanity.com/paper/2506.08906/) (2025)
- [Geometry-aware Distance Measure for Diverse Hierarchical Structures in Hyperbolic Spaces](https://mlsanity.com/paper/2506.18533/) (2025)
- [EEML: Ensemble Embedded Meta-learning](https://mlsanity.com/paper/2206.09195/) (2022)
- [AI Act for the Working Programmer](https://mlsanity.com/paper/2408.01449/) (2024)
- [EfficientFSL: Enhancing Few-Shot Classification via Query-Only Tuning in Vision Transformers](https://mlsanity.com/paper/2601.08499/) (2026)

## Abstract (arXiv)

Few-shot learning is a challenging problem where the goal is to achieve generalization from only few examples. Model-agnostic meta-learning (MAML) tackles the problem by formulating prior knowledge as a common initialization across tasks, which is then used to quickly adapt to unseen tasks. However, forcibly sharing an initialization can lead to conflicts among tasks and the compromised (undesired by tasks) location on optimization landscape, thereby hindering the task adaptation. Further, we observe that the degree of conflict differs among not only tasks but also layers of a neural network. Thus, we propose task-and-layer-wise attenuation on the compromised initialization to reduce its influence. As the attenuation dynamically controls (or selectively forgets) the influence of prior knowledge for a given task and each layer, we name our method as L2F (Learn to Forget). The experimental results demonstrate that the proposed method provides faster adaptation and greatly improves the performance. Furthermore, L2F can be easily applied and improve other state-of-the-art MAML-based frameworks, illustrating its simplicity and generalizability.