# Metalearning with Hebbian Fast Weights

arXiv: https://arxiv.org/abs/1807.05076 · HTML: https://mlsanity.com/paper/1807.05076/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2018.

## Built on by (most cited first, 15 of 15)

- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [Kimi Linear: An Expressive, Efficient Attention Architecture](https://mlsanity.com/paper/2510.26692/) (2025)
- [A Modern Self-Referential Weight Matrix That Learns to Modify Itself](https://mlsanity.com/paper/2202.05780/) (2022)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Fast weight programming and linear transformers: from machine learning to neurobiology](https://mlsanity.com/paper/2508.08435/) (2025)
- [Metalearning Continual Learning Algorithms](https://mlsanity.com/paper/2312.00276/) (2023)
- [Towards Zero-shot Language Modeling](https://mlsanity.com/paper/2108.03334/) (2021)
- [Where to Bind Matters: Hebbian Fast Weights in Vision Transformers for Few-Shot Character Recognition](https://mlsanity.com/paper/2605.02920/) (2026)
- [EMO: Episodic Memory Optimization for Few-Shot Meta-Learning](https://mlsanity.com/paper/2306.05189/) (2023)
- [Adaptive Hebbian Memory Routing in Vision Transformers for Few-Shot Learning](https://mlsanity.com/paper/2606.24756/) (2026)
- [Memory-enriched computation and learning in spiking neural networks through Hebbian plasticity](https://mlsanity.com/paper/2205.11276/) (2022)
- [Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networks](https://mlsanity.com/paper/2211.09440/) (2022)
- [Where Should Experience Live? Hierarchical Hebbian Memory for Continual Vision Transformers](https://mlsanity.com/paper/2609.00358/) (2026)
- [Representation Memorization for Fast Learning New Knowledge without Forgetting](https://mlsanity.com/paper/2108.12596/) (2021)
- [Multi-Plasticity Synergy with Adaptive Mechanism Assignment for Training Spiking Neural Networks](https://mlsanity.com/paper/2508.13673/) (2025)

## Abstract (arXiv)

We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast weights constructed by a Hebbian learning rule implement one-shot binding for each new task. On the Omniglot, Mini-ImageNet, and Penn Treebank one-shot learning benchmarks, our model achieves state-of-the-art results.