mlsanityData to 7 Oct 2026

Knowledge Mechanisms in Large Language Models: A Survey and Perspective

arXiv 2407.15017 ↗First posted 22 Jul 2024cs.CL cs.AI cs.CVcited by 50

Not on the map. Why?

Abstract (arXiv)

Understanding knowledge mechanisms in Large Language Models (LLMs) is crucial for advancing towards trustworthy AGI. This paper reviews knowledge mechanism analysis from a novel taxonomy including knowledge utilization and evolution. Knowledge utilization delves into the mechanism of memorization, comprehension and application, and creation. Knowledge evolution focuses on the dynamic progression of knowledge within individual and group LLMs. Moreover, we discuss what knowledge LLMs have learned, the reasons for the fragility of parametric knowledge, and the potential dark knowledge (hypothesis) that will be challenging to address. We hope this work can help understand knowledge in LLMs and provide insights for future research.

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 212 of its 370 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

212 of 370 references.

References · 212

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  11. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  12. 2023A Survey of Large Language Modelscited by 2,537
38 more
  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  2. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  3. 2019Language Models as Knowledge Bases?cited by 1,922
  4. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  6. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  10. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  12. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  13. 2018Hierarchical Neural Story Generationcited by 1,182
  14. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  15. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  16. 2023LIMA: Less Is More for Alignmentcited by 1,080
  17. 2020Concept Bottleneck Modelscited by 1,022
  18. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  19. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  20. 2022In-context Learning and Induction Headscited by 930
  21. 2022Toy Models of Superpositioncited by 928
  22. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  23. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
  24. 2023Large Language Modelscited by 774
  25. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  26. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  27. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  28. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  29. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  30. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  31. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  32. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  33. 2021Editing Factual Knowledge in Language Modelscited by 548
  34. 2019ERNIE: Enhanced Language Representation with Informative Entitiescited by 525
  35. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  36. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
  37. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  38. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 452

50 most cited of 212 All 212 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 30; 2026: 7.

50 papers.

Sort:

By area:

  1. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  2. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  3. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  4. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  5. 2024Can Editing LLMs Inject Harm?cited by 22
  6. 2025How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Trainingcited by 22
  7. 2025Mask-DPO: Generalizable Fine-grained Factuality Alignment of LLMscited by 22
  8. 2024A Survey on the Honesty of Large Language Modelscited by 21
  9. 2024TMGBench: A Systematic Game Benchmark for Evaluating Strategic Reasoning Abilities of LLMscited by 21
  10. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimizationcited by 20
  11. 2025Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysiscited by 20
  12. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
38 more
  1. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
  2. 2024Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuningcited by 14
  3. 2025Language Models' Factuality Depends on the Language of Inquirycited by 14
  4. 2024TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Usecited by 11
  5. 2024Neuron-based Personality Trait Induction in Large Language Modelscited by 9
  6. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  7. 2025OntoTune: Ontology-Driven Self-training for Aligning Large Language Modelscited by 8
  8. 2026Why Steering Works: Toward a Unified View of Language Model Parameter Dynamicscited by 8
  9. 2024ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domainscited by 7
  10. 2025Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levelscited by 7
  11. 2025COMPKE: Complex Question Answering under Knowledge Editingcited by 6
  12. 2026From Data to Behavior: Predicting Unintended Model Behaviors Before Trainingcited by 6
  13. 2025Large Language Model Sourcing: A Surveycited by 5
  14. 2025Provable Benefits of In-Tool Learning for Large Language Modelscited by 4
  15. 2024One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Modelscited by 3
  16. 2025Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Trainingcited by 3
  17. 2025Synthesize-on-Graph: Knowledgeable Synthetic Data Generation for Continue Pre-training of Large Language Modelscited by 3
  18. 2025KScope: A Framework for Characterizing the Knowledge Status of Language Modelscited by 3
  19. 2025DSCD: Large Language Model Detoxification with Self-Constrained Decodingcited by 3
  20. 2025Generalization or Memorization: Dynamic Decoding for Mode Steeringcited by 3
  21. 2026Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activationcited by 3
  22. 202460 Data Points are Sufficient to Fine-Tune LLMs for Question-Answeringcited by 2
  23. 2025Concept Incongruence: An Exploration of Time and Death in Role Playingcited by 2
  24. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 2
  25. 2025Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Modelscited by 2
  26. 2025DFAMS: Dynamic-flow guided Federated Alignment based Multi-prototype Searchcited by 2
  27. 2025Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detectioncited by 2
  28. 2025Open Problems and a Hypothetical Path Forward in LLM Knowledge Paradigmscited by 1
  29. 2025Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Modelscited by 1
  30. 2025STEAM: A Semantic-Level Knowledge Editing Framework for Large Language Modelscited by 1
  31. 2025NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Modelscited by 1
  32. 2026How Do Large Language Models Learn Concepts During Continual Pre-Training?cited by 1
  33. 2025One for All: Update Parameterized Knowledge Across Multiple Modelscited by 0
  34. 2025Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Modelscited by 0
  35. 2025Evontree: Ontology Rule-Guided Self-Evolution of Large Language Modelscited by 0
  36. 2026Reliable AI Needs to Externalize Implicit Knowledge: A Human-AI Collaboration Perspectivecited by 0
  37. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modescited by 0
  38. 2026Geo-Spatial Concept Probing of Large Language Models: Abstraction, Compositionality, and Groundingcited by 0