mlsanityData to 7 Oct 2026

Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers

arXiv 2406.18400 ↗First posted 26 Jun 2024cs.CL cs.LG stat.MLcited by 14

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be easily manipulated by changing contexts, even without altering their factual meanings. These findings highlight that LLMs might behave like an associative memory model where certain tokens in the contexts serve as clues to retrieving facts. We mathematically explore this property by studying how transformers, the building blocks of LLMs, can complete such memory tasks. We study a simple latent concept association problem with a one-layer transformer and we show theoretically and empirically that the transformer gathers information using self-attention and uses the value matrix for associative memory.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 85 of its 102 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

85 of 102 references.

References · 85

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  8. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  9. 2022Mass-Editing Memory in a Transformercited by 976
  10. 2022In-context Learning and Induction Headscited by 930
  11. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  12. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
38 more
  1. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  2. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  5. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  7. 2021Editing Factual Knowledge in Language Modelscited by 548
  8. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  9. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  10. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  11. 2021Fast Model Editing at Scalecited by 491
  12. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  13. 2020Hopfield Networks is All You Needcited by 461
  14. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  15. 2022Memory-Based Model Editing at Scalecited by 443
  16. 2021Causal Abstractions of Neural Networkscited by 401
  17. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  18. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  19. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
  20. 2023Role-Play with Large Language Modelscited by 290
  21. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  22. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  23. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  24. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 229
  25. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  26. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  27. 2020How Context Affects Language Models' Factual Predictionscited by 181
  28. 2023Birth of a Transformer: A Memory Viewpointcited by 154
  29. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
  30. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
  31. 2023Physics of Language Models: Part 3.2, Knowledge Manipulationcited by 142
  32. 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 137
  33. 2024The Evolution of Statistical Induction Heads: In-Context Learning Markov Chainscited by 115
  34. 2024How Transformers Learn Causal Structure with Gradient Descentcited by 114
  35. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 111
  36. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  37. 2022Unveiling Transformers with LEGO: a synthetic reasoning taskcited by 105
  38. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102

50 most cited of 85 All 85 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 6; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  2. 2024Understanding Factual Recall in Transformers via Associative Memoriescited by 45
  3. 2025Taming Knowledge Conflicts in Language Modelscited by 27
  4. 2026Understanding LoRA as Knowledge Memory: An Empirical Analysiscited by 11
  5. 2025Strong Memory, Weak Control: An Empirical Study of Executive Functioning in LLMscited by 6
  6. 2025Stochastic Chameleons: Irrelevant Context Hallucinations Reveal Class-Based (Mis)Generalization in LLMscited by 5
  7. 2025On the Robustness of Transformers against Context Hijacking for Linear Classificationcited by 4
  8. 2025Pre-trained Large Language Models Learn Hidden Markov Models In-contextcited by 4
  9. 2024Associative memory inspires improvements for in-context learning using a novel attention residual stream architecturecited by 3
  10. 2025Large Language Models as Model Organisms for Human Associative Learningcited by 1
  11. 2024Rethinking Associative Memory Mechanism in Induction Headcited by 0
  12. 2026Where Does Robustness Live? Neuron-Guided Adaptation for Retrieval-Augmented Language Modelscited by 0
2 more
  1. 2026When Context Misleads: Surprisal, Energy and Attention Entropy as Metrics of Coherence Illusions in LLMscited by 0
  2. 2026Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibilitycited by 0