mlsanityData to 7 Oct 2026

Strategic resource allocation in memory encoding: An efficiency principle shaping language processing

arXiv 2503.14728 ↗First posted 18 Mar 2025cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

How is the limited capacity of working memory efficiently used to support human linguistic behaviors? In this paper, we propose Strategic Resource Allocation (SRA) as an efficiency principle for memory encoding in sentence processing. The idea is that working memory resources are dynamically and strategically allocated to prioritize novel and unexpected information. From a resource-rational perspective, we argue that SRA is the principled solution to a computational problem posed by two functional assumptions about working memory, namely its limited capacity and its noisy representation. Specifically, working memory needs to minimize the retrieval error of past inputs under the constraint of limited memory resources, an optimization problem whose solution is to allocate more resources to encode more surprising inputs with higher precision. One of the critical consequences of SRA is that surprising inputs are encoded with enhanced representations, and therefore are less susceptible to memory decay and interference. Empirically, through naturalistic corpus data, we find converging evidence for SRA in the context of dependency locality from both production and comprehension, where non-local dependencies with less predictable antecedents are associated with reduced locality effect. However, our results also reveal considerable cross-linguistic variability, suggesting the need for a closer examination of how SRA, as a domain-general memory efficiency principle, interacts with language-specific phrase structures. SRA highlights the critical role of representational uncertainty in understanding memory encoding. It also reimages the effects of surprisal and entropy on processing difficulty from the perspective of efficient memory encoding.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 150 references that are in the map. First posted on arXiv 18 Mar 2025.

Builds on

17 of 150 references.

References · 17

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Universal Dependencies v2: An Evergrowing Multilingual Treebank Collectioncited by 269
  3. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 163
  4. 2022mGPT: Few-Shot Learners Go Multilingualcited by 141
  5. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 115
  6. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 111
  7. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 75
  8. 2021Revisiting the Uniform Information Density Hypothesiscited by 63
  9. 2017The Natural Stories Corpuscited by 40
  10. 2020Probabilistic Predictions of People Perusing: Evaluating Metrics of Language Model Performance for Psycholinguistic Modelingcited by 22
  11. 2023A Language Model with Limited Memory Capacity Captures Interference in Human Sentence Processingcited by 22
  12. 2024On the Role of Context in Reading Time Predictioncited by 17
5 more
  1. 2023A Cross-Linguistic Pressure for Uniform Information Density in Word Ordercited by 16
  2. 2022Crosslinguistic word order variation reflects evolutionary pressures of dependency and information localitycited by 10
  3. 2023Revisiting the Optimality of Word Lengthscited by 5
  4. 2024A hierarchical Bayesian model for syntactic primingcited by 4
  5. 2022Enhancements to the BOUN Treebank Reflecting the Agglutinative Nature of Turkishcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.