# Meta-Curvature

arXiv: https://arxiv.org/abs/1902.03356 · HTML: https://mlsanity.com/paper/1902.03356/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2019.

## Built on by (most cited first, 30 of 70)

- [Transformers learn in-context by gradient descent](https://mlsanity.com/paper/2212.07677/) (2022)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [On Learning the Geodesic Path for Incremental Learning](https://mlsanity.com/paper/2104.08572/) (2021)
- [Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond](https://mlsanity.com/paper/2110.00455/) (2021)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models](https://mlsanity.com/paper/2211.14946/) (2022)
- [Rectifying the Shortcut Learning of Background for Few-Shot Learning](https://mlsanity.com/paper/2107.07746/) (2021)
- [A Closer Look at Few-shot Classification Again](https://mlsanity.com/paper/2301.12246/) (2023)
- [Meta-Learning with Fewer Tasks through Task Interpolation](https://mlsanity.com/paper/2106.02695/) (2021)
- [Bootstrapped Meta-Learning](https://mlsanity.com/paper/2109.04504/) (2021)
- [MetaDiff: Meta-Learning with Conditional Diffusion for Few-Shot Learning](https://mlsanity.com/paper/2307.16424/) (2023)
- [Learning where to learn: Gradient sparsity in meta and continual learning](https://mlsanity.com/paper/2110.14402/) (2021)
- [On Learning Fairness and Accuracy on Multiple Subgroups](https://mlsanity.com/paper/2210.10837/) (2022)
- [ORBIT: A Real-World Few-Shot Dataset for Teachable Object Recognition](https://mlsanity.com/paper/2104.03841/) (2021)
- [MetaAudio: A Few-Shot Audio Classification Benchmark](https://mlsanity.com/paper/2204.02121/) (2022)
- [Transductive Few-Shot Classification on the Oblique Manifold](https://mlsanity.com/paper/2108.04009/) (2021)
- [Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction](https://mlsanity.com/paper/2205.02708/) (2022)
- [Channel Importance Matters in Few-Shot Image Classification](https://mlsanity.com/paper/2206.08126/) (2022)
- [Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classification](https://mlsanity.com/paper/2208.12398/) (2022)
- [On sensitivity of meta-learning to support data](https://mlsanity.com/paper/2110.13953/) (2021)
- [Transformer-Based Learned Optimization](https://mlsanity.com/paper/2212.01055/) (2022)
- [Meta-Learning with a Geometry-Adaptive Preconditioner](https://mlsanity.com/paper/2304.01552/) (2023)
- [MetaDelta: A Meta-Learning System for Few-shot Image Classification](https://mlsanity.com/paper/2102.10744/) (2021)
- [Amortized Proximal Optimization](https://mlsanity.com/paper/2203.00089/) (2022)
- [Few-Shot Learning by Dimensionality Reduction in Gradient Space](https://mlsanity.com/paper/2206.03483/) (2022)
- [Stateless Neural Meta-Learning using Second-Order Gradients](https://mlsanity.com/paper/2104.10527/) (2021)
- [Meta Mirror Descent: Optimiser Learning for Fast Convergence](https://mlsanity.com/paper/2203.02711/) (2022)

## Abstract (arXiv)

We propose meta-curvature (MC), a framework to learn curvature information for better generalization and fast model adaptation. MC expands on the model-agnostic meta-learner (MAML) by learning to transform the gradients in the inner optimization such that the transformed gradients achieve better generalization performance to a new task. For training large scale neural networks, we decompose the curvature matrix into smaller matrices in a novel scheme where we capture the dependencies of the model's parameters with a series of tensor products. We demonstrate the effects of our proposed method on several few-shot learning tasks and datasets. Without any task specific techniques and architectures, the proposed method achieves substantial improvement upon previous MAML variants and outperforms the recent state-of-the-art methods. Furthermore, we observe faster convergence rates of the meta-training process. Finally, we present an analysis that explains better generalization performance with the meta-trained curvature.