mlsanityData to 14 Oct 2026

Self-Attentive Associative Memory

arXiv 2002.03519 ↗First posted 10 Feb 2020cs.LG stat.MLcited by 33

Not on the map. Why?

Abstract (arXiv)

Heretofore, neural networks with external memory are restricted to single memory with lossy representations of memory interactions. A rich representation of relationships between memory pieces urges a high-order and segregated relational memory. In this paper, we propose to separate the storage of individual experiences (item memory) and their occurring relationships (relational memory). The idea is implemented through a novel Self-attentive Associative Memory (SAM) operator. Found upon outer product, SAM forms a set of associative memories that represent the hypothetical high-order relationships between arbitrary pairs of memory elements, through which a relational memory is constructed from an item memory. The two memories are wired into a single sequential model capable of both memorization and relational reasoning. We achieve competitive results with our proposed two-memory model in a diversity of machine learning tasks, from challenging synthetic problems to practical testbeds such as geometry, graph, reinforcement learning, and question answering.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 8; 2023: 11; 2024: 6; 2025: 2.

33 papers.

Sort:

By area:

  1. 2024Titans: Learning to Memorize at Test Timecited by 328
  2. 2022StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Textscited by 68
  3. 2023Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Textcited by 59
  4. 2021ABC: Attention with Bounded-memory Controlcited by 36
  5. 2022Attention Mechanism in Neural Networks: Where it Comes and Where it Goescited by 29
  6. 2021Model-Based Episodic Memory Induces Dynamic Hybrid Controlscited by 14
  7. 2023Recasting Self-Attention with Holographic Reduced Representationscited by 14
  8. 2022A Call for Clarity in Beam Search: How It Works and When It Stopscited by 10
  9. 2023Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perceptioncited by 9
  10. 2024Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learningcited by 9
  11. 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
  12. 2023Memoria: Resolving Fateful Forgetting Problem through Human-Inspired Memory Architecturecited by 8
21 more
  1. 2021Dyna-bAbI: unlocking bAbI's potential with dynamic synthetic benchmarkingcited by 7
  2. 2021Learning to Rehearse in Long Sequence Memorizationcited by 6
  3. 2022Short-Term Plasticity Neurons Learning to Learn and Forgetcited by 6
  4. 2022Graph Neural Networks Extract High-Resolution Cultivated Land Maps from Sentinel-2 Image Seriescited by 6
  5. 2021HIT: A Hierarchically Fused Deep Attention Network for Robust Code-mixed Language Representationcited by 5
  6. 2021Episodic Policy Gradient Trainingcited by 5
  7. 2023Learning Language Representations with Logical Inductive Biascited by 5
  8. 2023Persona-aware Generative Model for Code-mixed Languagecited by 3
  9. 2023DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Textcited by 3
  10. 2022A Comprehensive Understanding of Code-mixed Language Semantics using Hierarchical Transformercited by 2
  11. 2022Memory-Augmented Graph Neural Networks: A Brain-Inspired Reviewcited by 2
  12. 2023A Brain-inspired Memory Transformation based Differentiable Neural Computer for Reasoning-based Question Answeringcited by 2
  13. 2023A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Informationcited by 2
  14. 2023Knowledge Authoring for Rules and Actionscited by 2
  15. 2023A Framework for Inference Inspired by Human Memory Mechanismscited by 2
  16. 2024Attention-based Iterative Decomposition for Tensor Product Representationcited by 2
  17. 2025Re:Frame -- Retrieving Experience From Associative Memorycited by 1
  18. 2022Bird-Eye Transformers for Text Generation Modelscited by 0
  19. 2024Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memorycited by 0
  20. 2024Entropic associative memory for real world imagescited by 0
  21. 2024Discrete Dictionary-based Decomposition Layer for Structured Representation Learningcited by 0