# Meta Networks

arXiv: https://arxiv.org/abs/1703.00837 · HTML: https://mlsanity.com/paper/1703.00837/

## Key facts

- Cited by 269 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2017.

## Built on by (most cited first, 30 of 269)

- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention](https://mlsanity.com/paper/2404.07143/) (2024)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [Learning from Few Examples: A Summary of Approaches to Few-Shot Learning](https://mlsanity.com/paper/2203.04291/) (2022)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [Learning Normal Dynamics in Videos with Meta Prototype Network](https://mlsanity.com/paper/2104.06689/) (2021)
- [Mining Latent Classes for Few-shot Segmentation](https://mlsanity.com/paper/2103.15402/) (2021)
- [Advances and Challenges in Meta-Learning: A Technical Review](https://mlsanity.com/paper/2307.04722/) (2023)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [Transferability in Deep Learning: A Survey](https://mlsanity.com/paper/2201.05867/) (2022)
- [Learning Meta-class Memory for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2108.02958/) (2021)
- [Omni-Dimensional Dynamic Convolution](https://mlsanity.com/paper/2209.07947/) (2022)
- [Multiple Meta-model Quantifying for Medical Visual Question Answering](https://mlsanity.com/paper/2105.08913/) (2021)
- [Pareto Self-Supervised Training for Few-Shot Learning](https://mlsanity.com/paper/2104.07841/) (2021)
- [Rectifying the Shortcut Learning of Background for Few-Shot Learning](https://mlsanity.com/paper/2107.07746/) (2021)
- [Meta-Inverse Reinforcement Learning with Probabilistic Context Variables](https://mlsanity.com/paper/1909.09314/) (2019)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [A Survey on Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2304.08130/) (2023)
- [A Closer Look at Few-shot Classification Again](https://mlsanity.com/paper/2301.12246/) (2023)
- [Hypernetworks in Meta-Reinforcement Learning](https://mlsanity.com/paper/2210.11348/) (2022)
- [A Modern Self-Referential Weight Matrix That Learns to Modify Itself](https://mlsanity.com/paper/2202.05780/) (2022)
- [Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Concepts](https://mlsanity.com/paper/2410.12777/) (2024)
- [Meta-Learning Dynamics Forecasting Using Task Inference](https://mlsanity.com/paper/2102.10271/) (2021)
- [Learning Domain Invariant Prompt for Vision-Language Models](https://mlsanity.com/paper/2212.04196/) (2022)
- [A Metalearning Approach for Physics-Informed Neural Networks (PINNs): Application to Parameterized PDEs](https://mlsanity.com/paper/2110.13361/) (2021)
- [On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2102.10454/) (2021)
- [Learning Implicit Temporal Alignment for Few-shot Video Classification](https://mlsanity.com/paper/2105.04823/) (2021)

## Abstract (arXiv)

Neural networks have been successfully applied in applications with a large amount of labeled data. However, the task of rapid generalization on new concepts with small training data while preserving performances on previously learned ones still presents a significant challenge to neural network models. In this work, we introduce a novel meta learning method, Meta Networks (MetaNet), that learns a meta-level knowledge across tasks and shifts its inductive biases via fast parameterization for rapid generalization. When evaluated on Omniglot and Mini-ImageNet benchmarks, our MetaNet models achieve a near human-level performance and outperform the baseline approaches by up to 6% accuracy. We demonstrate several appealing properties of MetaNet relating to generalization and continual learning.