# A Bridge Between Hyperparameter Optimization and Learning-to-learn

arXiv: https://arxiv.org/abs/1712.06283 · HTML: https://mlsanity.com/paper/1712.06283/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Dec 2017.

## Built on by (most cited first, 6 of 6)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [How to Train Your MAML to Excel in Few-Shot Classification](https://mlsanity.com/paper/2106.16245/) (2021)
- [Hyper-parameter tuning of physics-informed neural networks: Application to Helmholtz problems](https://mlsanity.com/paper/2205.06704/) (2022)
- [iSLAM: Imperative SLAM](https://mlsanity.com/paper/2306.07894/) (2023)
- [Learning with Constraint Learning: New Perspective, Solution Strategy and Various Applications](https://mlsanity.com/paper/2307.15257/) (2023)
- [Dual Meta-Learning with Longitudinally Generalized Regularization for One-Shot Brain Tissue Segmentation Across the Human Lifespan](https://mlsanity.com/paper/2308.06774/) (2023)

## Abstract (arXiv)

We consider a class of a nested optimization problems involving inner and outer objectives. We observe that by taking into explicit account the optimization dynamics for the inner objective it is possible to derive a general framework that unifies gradient-based hyperparameter optimization and meta-learning (or learning-to-learn). Depending on the specific setting, the variables of the outer objective take either the meaning of hyperparameters in a supervised learning problem or parameters of a meta-learner. We show that some recently proposed methods in the latter setting can be instantiated in our framework and tackled with the same gradient-based algorithms. Finally, we discuss possible design patterns for learning-to-learn and present encouraging preliminary experiments for few-shot learning.