mlsanityData to 7 Oct 2026

Beyond Memorization: The Challenge of Random Memory Access in Language Models

arXiv 2403.07805 ↗First posted 12 Mar 2024cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent developments in Language Models (LMs) have shown their effectiveness in NLP tasks, particularly in knowledge-intensive tasks. However, the mechanisms underlying knowledge storage and memory access within their parameters remain elusive. In this paper, we investigate whether a generative LM (e.g., GPT-2) is able to access its memory sequentially or randomly. Through carefully-designed synthetic tasks, covering the scenarios of full recitation, selective recitation and grounded question answering, we reveal that LMs manage to sequentially access their memory while encountering challenges in randomly accessing memorized content. We find that techniques including recitation and permutation improve the random memory access capability of LMs. Furthermore, by applying this intervention to realistic scenarios of open-domain question answering, we validate that enhancing random access by recitation leads to notable improvements in question answering. The code to reproduce our experiments can be found at https://github.com/sail-sg/lm-random-memory-access.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 41 of its 64 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

41 of 64 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2020Extracting Training Data from Large Language Modelscited by 2,144
  11. 2019Language Models as Knowledge Bases?cited by 1,922
  12. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
29 more
  1. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  2. 2023LIMA: Less Is More for Alignmentcited by 1,080
  3. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  4. 2020KILT: a Benchmark for Knowledge Intensive Language Taskscited by 571
  5. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 439
  6. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  7. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  8. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  9. 2022Transformer Memory as a Differentiable Search Indexcited by 242
  10. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  11. 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 195
  12. 2023Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMscited by 175
  13. 2022A Review on Language Models as Knowledge Basescited by 156
  14. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  15. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  16. 2020Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queriescited by 107
  17. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  18. 2022A Neural Corpus Indexer for Document Retrievalcited by 107
  19. 2024RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculturecited by 80
  20. 2019Inducing Relational Knowledge from BERTcited by 75
  21. 2021Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA?cited by 62
  22. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 47
  23. 2023How Does Generative Retrieval Scale to Millions of Passages?cited by 47
  24. 2023Scalable and Effective Generative Information Retrievalcited by 38
  25. 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
  26. 2023Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Modelscited by 32
  27. 2023Large Language Models are Built-in Autoregressive Search Enginescited by 26
  28. 2023TOME: A Two-stage Approach for Model-based Retrievalcited by 21
  29. 2023GLEN: Generative Retrieval via Lexical Index Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 1.

8 papers.

  1. 2024Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Modelscited by 18
  2. 2025From Style to Facts: Mapping the Boundaries of Knowledge Injection with Finetuningcited by 7
  3. 2025Large Language Models Penetration in Scholarly Writing and Peer Reviewcited by 6
  4. 2025A Review of Incorporating Psychological Theories in LLMscited by 5
  5. 2025Memorizing is Not Enough: Deep Knowledge Injection Through Reasoningcited by 4
  6. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  7. 2025Out-of-Context Reasoning in Large Language Modelscited by 2
  8. 2026How LoRA Remembers? A Parametric Memory Law for LLM Finetuningcited by 2