# A Simple Neural Attentive Meta-Learner

arXiv: https://arxiv.org/abs/1707.03141 · HTML: https://mlsanity.com/paper/1707.03141/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2017.

## Built on by (most cited first, 24 of 24)

- [Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection](https://mlsanity.com/paper/2306.04637/) (2023)
- [Offline Meta-Reinforcement Learning with Advantage Weighting](https://mlsanity.com/paper/2008.06043/) (2020)
- [Partial Is Better Than All: Revisiting Fine-tuning Strategy for Few-shot Learning](https://mlsanity.com/paper/2102.03983/) (2021)
- [LEADS: Learning Dynamical Systems that Generalize Across Environments](https://mlsanity.com/paper/2106.04546/) (2021)
- [Learning a Prior over Intent via Meta-Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.12573/) (2018)
- [Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning](https://mlsanity.com/paper/2402.02429/) (2024)
- [On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2210.10763/) (2022)
- [Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysis](https://mlsanity.com/paper/2208.03051/) (2022)
- [FOCAL: Efficient Fully-Offline Meta-Reinforcement Learning via Distance Metric Learning and Behavior Regularization](https://mlsanity.com/paper/2010.01112/) (2020)
- [Improved Meta Learning for Low Resource Speech Recognition](https://mlsanity.com/paper/2205.06182/) (2022)
- [On the Opportunities of Green Computing: A Survey](https://mlsanity.com/paper/2311.00447/) (2023)
- [Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning](https://mlsanity.com/paper/2201.05151/) (2022)
- [Inductive Linear Probing for Few-shot Node Classification](https://mlsanity.com/paper/2306.08192/) (2023)
- [Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables](https://mlsanity.com/paper/2509.03845/) (2025)
- [Learning More Discriminative Local Descriptors for Few-shot Learning](https://mlsanity.com/paper/2305.08721/) (2023)
- [Task-Specific Preconditioner for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2412.15483/) (2024)
- [Transductive One-Shot Learning Meet Subspace Decomposition](https://mlsanity.com/paper/2504.00348/) (2025)
- [SMGI: A Structural Theory of General Artificial Intelligence](https://mlsanity.com/paper/2603.07896/) (2026)
- [In-Context Learning for Robots: Methods and Applications](https://mlsanity.com/paper/2609.36012/) (2026)
- [Accelerating Gradient-based Meta Learner](https://mlsanity.com/paper/2110.14459/) (2021)
- [Self-Adaptive Label Augmentation for Semi-supervised Few-shot Classification](https://mlsanity.com/paper/2206.08150/) (2022)
- [Neural Routing in Meta Learning](https://mlsanity.com/paper/2210.07932/) (2022)
- [Adaptive Growth: Real-time CNN Layer Expansion](https://mlsanity.com/paper/2309.03049/) (2023)
- [Visually Grounded Speech Models for Low-resource Languages and Cognitive Modelling](https://mlsanity.com/paper/2409.02865/) (2024)

## Abstract (arXiv)

Deep neural networks excel in regimes with large amounts of data, but tend to struggle when data is scarce or when they need to adapt quickly to changes in the task. In response, recent work in meta-learning proposes training a meta-learner on a distribution of similar tasks, in the hopes of generalization to novel but related tasks by learning a high-level strategy that captures the essence of the problem it is asked to solve. However, many recent meta-learning approaches are extensively hand-designed, either using architectures specialized to a particular application, or hard-coding algorithmic components that constrain how the meta-learner solves the task. We propose a class of simple and generic meta-learner architectures that use a novel combination of temporal convolutions and soft attention; the former to aggregate information from past experience and the latter to pinpoint specific pieces of information. In the most extensive set of meta-learning experiments to date, we evaluate the resulting Simple Neural AttentIve Learner (or SNAIL) on several heavily-benchmarked tasks. On all tasks, in both supervised and reinforcement learning, SNAIL attains state-of-the-art performance by significant margins.