# Meta-learning the Learning Trends Shared Across Tasks

arXiv: https://arxiv.org/abs/2010.09291 · HTML: https://mlsanity.com/paper/2010.09291/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [HyperShot: Few-Shot Learning by Kernel HyperNetworks](https://mlsanity.com/paper/2203.11378/) (2022)
- [HyperMAML: Few-Shot Adaptation of Deep Models with Hypernetworks](https://mlsanity.com/paper/2205.15745/) (2022)
- [Meta-Learning with a Geometry-Adaptive Preconditioner](https://mlsanity.com/paper/2304.01552/) (2023)
- [FewShotNeRF: Meta-Learning-based Novel View Synthesis for Rapid Scene-Specific Adaptation](https://mlsanity.com/paper/2408.04803/) (2024)
- [Task-Specific Preconditioner for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2412.15483/) (2024)
- [Hypernetwork approach to Bayesian MAML](https://mlsanity.com/paper/2210.02796/) (2022)
- [Meta-Shop: Improving Item Advertisement For Small Businesses](https://mlsanity.com/paper/2212.01414/) (2022)
- [Learnable Loss Geometries with Mirror Descent for Scalable and Convergent Meta-Learning](https://mlsanity.com/paper/2509.02418/) (2025)

## Abstract (arXiv)

Meta-learning stands for 'learning to learn' such that generalization to new tasks is achieved. Among these methods, Gradient-based meta-learning algorithms are a specific sub-class that excel at quick adaptation to new tasks with limited data. This demonstrates their ability to acquire transferable knowledge, a capability that is central to human learning. However, the existing meta-learning approaches only depend on the current task information during the adaptation, and do not share the meta-knowledge of how a similar task has been adapted before. To address this gap, we propose a 'Path-aware' model-agnostic meta-learning approach. Specifically, our approach not only learns a good initialization for adaptation, it also learns an optimal way to adapt these parameters to a set of task-specific parameters, with learnable update directions, learning rates and, most importantly, the way updates evolve over different time-steps. Compared to the existing meta-learning methods, our approach offers: (a) The ability to learn gradient-preconditioning at different time-steps of the inner-loop, thereby modeling the dynamic learning behavior shared across tasks, and (b) The capability of aggregating the learning context through the provision of direct gradient-skip connections from the old time-steps, thus avoiding overfitting and improving generalization. In essence, our approach not only learns a transferable initialization, but also models the optimal update directions, learning rates, and task-specific learning trends. Specifically, in terms of learning trends, our approach determines the way update directions shape up as the task-specific learning progresses and how the previous update history helps in the current update. Our approach is simple to implement and demonstrates faster convergence. We report significant performance improvements on a number of FSL datasets.