# If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?

arXiv: https://arxiv.org/abs/2502.11469 · HTML: https://mlsanity.com/paper/2502.11469/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 69 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Constituency Parsing with a Self-Attentive Encoder](https://mlsanity.com/paper/1805.01052/) (2018)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior](https://mlsanity.com/paper/2006.01912/) (2020)
- [Recurrent Neural Network Grammars](https://mlsanity.com/paper/1602.07776/) (2016)
- [Human Sentence Processing: Recurrence or Attention?](https://mlsanity.com/paper/2005.09471/) (2020)
- [Finding Syntax in Human Encephalography with Beam Search](https://mlsanity.com/paper/1806.04127/) (2018)
- [Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilities](https://mlsanity.com/paper/2406.10851/) (2024)
- [Accounting for Agreement Phenomena in Sentence Comprehension with Transformer Language Models: Effects of Similarity-based Interference on Surprisal and Attention](https://mlsanity.com/paper/2104.12874/) (2021)
- [Effective Inference for Generative Neural Parsing](https://mlsanity.com/paper/1707.08976/) (2017)
- [A Language Model with Limited Memory Capacity Captures Interference in Human Sentence Processing](https://mlsanity.com/paper/2310.16142/) (2023)
- [What Do Recurrent Neural Network Grammars Learn About Syntax?](https://mlsanity.com/paper/1611.05774/) (2016)
- [Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?](https://mlsanity.com/paper/2107.09648/) (2021)
- [Entropy- and Distance-Based Predictors From GPT-2 Attention Patterns Predict Reading Times Over and Above GPT-2 Surprisal](https://mlsanity.com/paper/2212.11185/) (2022)
- [On the Proper Treatment of Tokenization in Psycholinguistics](https://mlsanity.com/paper/2410.02691/) (2024)
- [Effective Batching for Recurrent Neural Network Grammars](https://mlsanity.com/paper/2105.14822/) (2021)

## Built on by (most cited first, 4 of 4)

- [PathMem: Toward Cognition-Aligned Memory Transformation for Pathology MLLMs](https://mlsanity.com/paper/2603.09943/) (2026)
- [Memory Retrieval in Transformers: Insights from The Encoding Specificity Principle](https://mlsanity.com/paper/2601.20282/) (2026)
- [A Dual-Task Paradigm to Investigate Sentence Comprehension Strategies in Language Models](https://mlsanity.com/paper/2604.26351/) (2026)
- [Timesteps of Mamba Align with Human Reading Times](https://mlsanity.com/paper/2606.29904/) (2026)

## Abstract (arXiv)

Recent work in computational psycholinguistics has revealed intriguing parallels between attention mechanisms and human memory retrieval, focusing primarily on vanilla Transformers that operate on token-level representations. However, computational psycholinguistic research has also established that syntactic structures provide compelling explanations for human sentence processing that token-level factors cannot fully account for. In this paper, we investigate whether the attention mechanism of Transformer Grammar (TG), which uniquely operates on syntactic structures as representational units, can serve as a cognitive model of human memory retrieval, using Normalized Attention Entropy (NAE) as a linking hypothesis between models and humans. Our experiments demonstrate that TG's attention achieves superior predictive power for self-paced reading times compared to vanilla Transformer's, with further analyses revealing independent contributions from both models. These findings suggest that human sentence processing involves dual memory representations -- one based on syntactic structures and another on token sequences -- with attention serving as the general memory retrieval algorithm, while highlighting the importance of incorporating syntactic structures as representational units.