# Meta-Learning of Neural Architectures for Few-Shot Learning

arXiv: https://arxiv.org/abs/1911.11090 · HTML: https://mlsanity.com/paper/1911.11090/

## Key facts

- Cited by 73 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2019.

## Built on by (most cited first, 30 of 73)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges](https://mlsanity.com/paper/2107.05847/) (2021)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [Parameter Prediction for Unseen Deep Architectures](https://mlsanity.com/paper/2110.13100/) (2021)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharing](https://mlsanity.com/paper/2106.04502/) (2021)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets](https://mlsanity.com/paper/2107.00860/) (2021)
- [Physics-Informed Machine Learning for Modeling and Control of Dynamical Systems](https://mlsanity.com/paper/2306.13867/) (2023)
- [Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity](https://mlsanity.com/paper/2302.03407/) (2023)
- [AcroFOD: An Adaptive Method for Cross-domain Few-shot Object Detection](https://mlsanity.com/paper/2209.10904/) (2022)
- [Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning](https://mlsanity.com/paper/2203.09137/) (2022)
- [Temporal-Viewpoint Transportation Plan for Skeletal Few-shot Action Recognition](https://mlsanity.com/paper/2210.16820/) (2022)
- [Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learning](https://mlsanity.com/paper/2109.05749/) (2021)
- [Multimodal Parameter-Efficient Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2303.04751/) (2023)
- [What Matters For Meta-Learning Vision Regression Tasks?](https://mlsanity.com/paper/2203.04905/) (2022)
- [Design Principle Transfer in Neural Architecture Search via Large Language Models](https://mlsanity.com/paper/2408.11330/) (2024)
- [HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learning](https://mlsanity.com/paper/2106.08630/) (2021)
- [Constrained Bi-Level Optimization: Proximal Lagrangian Value function Approach and Hessian-free Algorithm](https://mlsanity.com/paper/2401.16164/) (2024)
- [Moreau Envelope for Nonconvex Bi-Level Optimization: A Single-loop and Hessian-free Solution Strategy](https://mlsanity.com/paper/2405.09927/) (2024)
- [Revisiting the Spatial and Temporal Modeling for Few-shot Action Recognition](https://mlsanity.com/paper/2301.07944/) (2023)
- [Simple Semantic-Aided Few-Shot Learning](https://mlsanity.com/paper/2311.18649/) (2023)
- [Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD](https://mlsanity.com/paper/2109.07497/) (2021)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Learnable Distribution Calibration for Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2210.00232/) (2022)
- [CrackNex: a Few-shot Low-light Crack Segmentation Model Based on Retinex Theory for UAV Inspections](https://mlsanity.com/paper/2403.03063/) (2024)
- [Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learning](https://mlsanity.com/paper/2104.02281/) (2021)
- [ECKPN: Explicit Class Knowledge Propagation Network for Transductive Few-shot Learning](https://mlsanity.com/paper/2106.08523/) (2021)
- [3D Skeleton-based Few-shot Action Recognition with JEANIE is not so Na"ive](https://mlsanity.com/paper/2112.12668/) (2021)

## Abstract (arXiv)

The recent progress in neural architecture search (NAS) has allowed scaling the automated design of neural architectures to real-world domains, such as object detection and semantic segmentation. However, one prerequisite for the application of NAS are large amounts of labeled data and compute resources. This renders its application challenging in few-shot learning scenarios, where many related tasks need to be learned, each with limited amounts of data and compute time. Thus, few-shot learning is typically done with a fixed neural architecture. To improve upon this, we propose MetaNAS, the first method which fully integrates NAS with gradient-based meta-learning. MetaNAS optimizes a meta-architecture along with the meta-weights during meta-training. During meta-testing, architectures can be adapted to a novel task with a few steps of the task optimizer, that is: task adaptation becomes computationally cheap and requires only little data per task. Moreover, MetaNAS is agnostic in that it can be used with arbitrary model-agnostic meta-learning algorithms and arbitrary gradient-based NAS methods. %We present encouraging results for MetaNAS with a combination of DARTS and REPTILE on few-shot classification benchmarks. Empirical results on standard few-shot classification benchmarks show that MetaNAS with a combination of DARTS and REPTILE yields state-of-the-art results.