# Meta-Learning with Network Pruning

arXiv: https://arxiv.org/abs/2007.03219 · HTML: https://mlsanity.com/paper/2007.03219/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2020.

## Built on by (most cited first, 14 of 14)

- [Meta-Learning Sparse Implicit Neural Representations](https://mlsanity.com/paper/2110.14678/) (2021)
- [Meta-Learning Sparse Compression Networks](https://mlsanity.com/paper/2205.08957/) (2022)
- [Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm](https://mlsanity.com/paper/2110.08190/) (2021)
- [Stability and Risk Bounds of Iterative Hard Thresholding](https://mlsanity.com/paper/2203.09413/) (2022)
- [Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks](https://mlsanity.com/paper/2205.15619/) (2022)
- [Dynamic Kernel Selection for Improved Generalization and Memory Efficiency in Meta-learning](https://mlsanity.com/paper/2206.01690/) (2022)
- [One-Shot Pruning for Fast-adapting Pre-trained Models on Devices](https://mlsanity.com/paper/2307.04365/) (2023)
- [B-SMALL: A Bayesian Neural Network approach to Sparse Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2101.00203/) (2021)
- [Voting from Nearest Tasks: Meta-Vote Pruning of Pre-trained Models for Downstream Tasks](https://mlsanity.com/paper/2301.11560/) (2023)
- [Learning to Learn with Indispensable Connections](https://mlsanity.com/paper/2304.02862/) (2023)
- [Improving Generalization in Meta-Learning via Meta-Gradient Augmentation](https://mlsanity.com/paper/2306.08460/) (2023)
- [Perturbing the Gradient for Alleviating Meta Overfitting](https://mlsanity.com/paper/2405.12299/) (2024)
- [Neuromodulated Meta-Learning](https://mlsanity.com/paper/2411.06746/) (2024)
- [Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning](https://mlsanity.com/paper/2509.02418/) (2025)

## Abstract (arXiv)

Meta-learning is a powerful paradigm for few-shot learning. Although with remarkable success witnessed in many applications, the existing optimization based meta-learning models with over-parameterized neural networks have been evidenced to ovetfit on training tasks. To remedy this deficiency, we propose a network pruning based meta-learning approach for overfitting reduction via explicitly controlling the capacity of network. A uniform concentration analysis reveals the benefit of network capacity constraint for reducing generalization gap of the proposed meta-learner. We have implemented our approach on top of Reptile assembled with two network pruning routines: Dense-Sparse-Dense (DSD) and Iterative Hard Thresholding (IHT). Extensive experimental results on benchmark datasets with different over-parameterized deep networks demonstrate that our method not only effectively alleviates meta-overfitting but also in many cases improves the overall generalization performance when applied to few-shot classification tasks.