mlsanityData to 7 Oct 2026

Learning Associative Inference Using Fast Weight Memory

arXiv 2011.07831 ↗First posted 16 Nov 2020cs.LG cs.NEcited by 51

Not on the map. Why?

Abstract (arXiv)

Humans can quickly associate stimuli to solve problems in novel contexts. Our novel neural network model learns state representations of facts that can be composed to perform such associative inference. To this end, we augment the LSTM model with an associative memory, dubbed Fast Weight Memory (FWM). Through differentiable operations at every step of a given input sequence, the LSTM updates and maintains compositional associations stored in the rapidly changing FWM weights. Our model is trained end-to-end by gradient descent and yields excellent performance on compositional language reasoning problems, meta-reinforcement-learning for POMDPs, and small-scale word-level language modelling.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 7; 2023: 7; 2024: 12; 2025: 13; 2026: 5.

51 papers.

Sort:

By area:

  1. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  2. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  3. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  4. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  5. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 169
  6. 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvaluescited by 97
  7. 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
  8. 2024Gated Slot Attention for Efficient Linear-Time Sequence Modelingcited by 73
  9. 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 69
  10. 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Textscited by 65
  11. 2025End-to-End Test-Time Training for Long Contextcited by 53
  12. 2025MesaNet: Sequence Modeling by Locally Optimal Test-Time Trainingcited by 37
38 more
  1. 2021FMMformer: Efficient and Flexible Transformer via Decomposed Near-field and Far-field Attentioncited by 26
  2. 2022Fine-Tuning Pre-trained Transformers into Decaying Fast Weightscited by 26
  3. 2021Introducing Symmetries to Black Box Meta Reinforcement Learningcited by 25
  4. 2021Fast Contextual Adaptation with Neural Associative Memory for On-Device Personalized Speech Recognitioncited by 25
  5. 2025Implicit Language Models are RNNs: Balancing Parallelization and Expressivitycited by 21
  6. 2025Structured Sparse Transition Matrices to Enable State Tracking in State-Space Modelscited by 15
  7. 2022Eliminating Meta Optimization Through Self-Referential Meta Learningcited by 12
  8. 2025Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformerscited by 12
  9. 2021Adaptive Convolutions with Per-pixel Dynamic Filter Atomcited by 11
  10. 2026Fast-weight Product Key Memorycited by 11
  11. 2022Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rulescited by 9
  12. 2024Consecutive Batch Model Editing with HooK Layerscited by 9
  13. 2021Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarkingcited by 7
  14. 2024Writing in the Margins: Better Inference Pattern for Long Context Retrievalcited by 7
  15. 2024Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentionscited by 6
  16. 2025Fast weight programming and linear transformers: from machine learning to neurobiologycited by 6
  17. 2025Fixed-Point RNNs: Interpolating from Diagonal to Densecited by 5
  18. 2026Learning State-Tracking from Code Using Linear RNNscited by 5
  19. 2022Meta-Reinforcement Learning with Self-Modifying Networkscited by 4
  20. 2023Knowledge-enhanced Neural Machine Reasoning: A Reviewcited by 3
  21. 2023Hebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNscited by 3
  22. 2023DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Textcited by 3
  23. 2025ParallelFlow: Parallelizing Linear Transformers via Flow Discretizationcited by 3
  24. 2025Beyond Sliding Windows: Learning to Manage Memory in Non-Markovian Environmentscited by 3
  25. 2023A Framework for Inference Inspired by Human Memory Mechanismscited by 2
  26. 2024Attention-based Iterative Decomposition for Tensor Product Representationcited by 2
  27. 2024IDEAL: Leveraging Infinite and Dynamic Characterizations of Large Language Models for Query-focused Summarizationcited by 2
  28. 2022DecBERT: Enhancing the Language Understanding of BERT with Causal Attention Maskscited by 1
  29. 2022Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networkscited by 1
  30. 2023Delving Deeper Into Astromorphic Transformerscited by 1
  31. 2025Re:Frame -- Retrieving Experience From Associative Memorycited by 1
  32. 2025Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticitycited by 1
  33. 2026Dissecting Linear Recurrent Models: How Different Gating Strategies Drive Selectivity and Generalizationcited by 1
  34. 2023Extending Memory for Language Modellingcited by 0
  35. 2023Advancing State of the Art in Language Modelingcited by 0
  36. 2024Discrete Dictionary-based Decomposition Layer for Structured Representation Learningcited by 0
  37. 2024Life, uh, Finds a Way: Hyperadaptability by Behavioral Searchcited by 0
  38. 2026Gecko: An Efficient Neural Architecture Inherently Processing Sequences with Arbitrary Lengthscited by 0

50 most cited of 51 All 51 (JSON)