mlsanityData to 14 Oct 2026

Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons

arXiv 2308.13198 ↗First posted 25 Aug 2023cs.CLcited by 56

Not on the map. Why?

Abstract (arXiv)

Pre-trained language models (PLMs) contain vast amounts of factual knowledge, but how the knowledge is stored in the parameters remains unclear. This paper delves into the complex task of understanding how factual knowledge is stored in multilingual PLMs, and introduces the Architecture-adapted Multilingual Integrated Gradients method, which successfully localizes knowledge neurons more precisely compared to current methods, and is more universal across various architectures and languages. Moreover, we conduct an in-depth exploration of knowledge neurons, leading to the following two important discoveries: (1) The discovery of Language-Independent Knowledge Neurons, which store factual knowledge in a form that transcends language. We design cross-lingual knowledge editing experiments, demonstrating that the PLMs can accomplish this task based on language-independent neurons; (2) The discovery of Degenerate Knowledge Neurons, a novel type of neuron showing that different knowledge neurons can store the same fact. Its property of functional overlap endows the PLMs with a robust mastery of factual knowledge. We design fact-checking experiments, proving that the degenerate knowledge neurons can help the PLMs to detect wrong facts. Experiments corroborate these findings, shedding light on the mechanisms of factual knowledge storage in multilingual PLMs, and contribute valuable insights to the field. The code is available at https://github.com/heng840/AMIG.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

27 of 38 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2023A Survey of Large Language Modelscited by 2,576
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  9. 2022Mass-Editing Memory in a Transformercited by 1,001
  10. 2019How Can We Know What Language Models Know?cited by 959
  11. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  12. 2021Fast Model Editing at Scalecited by 498
15 more
  1. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  2. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  3. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  4. 2020How Context Affects Language Models' Factual Predictionscited by 181
  5. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  6. 2021Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Modelscited by 132
  7. 2022Language Models as Agent Modelscited by 132
  8. 2022A Survey of Knowledge Enhanced Pre-trained Language Modelscited by 76
  9. 2022A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributionscited by 64
  10. 2023Scientific Fact-Checking: A Survey of Resources and Approachescited by 54
  11. 2023Sequential Integrated Gradients: a simple but effective method for explaining language modelscited by 52
  12. 2021Discretized Integrated Gradients for Explaining Language Modelscited by 46
  13. 2022How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysiscited by 42
  14. 2022Language Anisotropic Cross-Lingual Model Editingcited by 26
  15. 2023The Life Cycle of Knowledge in Big Language Models: A Surveycited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 30; 2025: 15; 2026: 7.

56 papers.

Sort:

By area:

  1. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 281
  2. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  3. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  4. 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
  5. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  6. 2024On the Multilingual Ability of Decoder-based Pre-trained Language Models: Finding and Controlling Language-Specific Neuronscited by 83
  7. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  8. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  9. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  10. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Modelscited by 44
  11. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  12. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
38 more
  1. 2025ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfercited by 29
  2. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  3. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  4. 2024MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Modelcited by 26
  5. 2024Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neuronscited by 25
  6. 2024Repetition Neurons: How Do Language Models Produce Repetitions?cited by 23
  7. 2024Can Editing LLMs Inject Harm?cited by 22
  8. 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 21
  9. 2024Multilingual Knowledge Editing with Language-Agnostic Factual Neuronscited by 18
  10. 2024Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editingcited by 18
  11. 2024AcTracer: Active Testing of Large Language Model via Multi-Stage Samplingcited by 17
  12. 2024Identifying Query-Relevant Neurons in Large Language Models for Long-Form Textscited by 14
  13. 2023Retrieval-augmented Multilingual Knowledge Editingcited by 13
  14. 2024Knowledge Localization: Mission Not Accomplished? Enter Query Localization!cited by 13
  15. 2025Less, but Better: Efficient Multilingual Expansion for LLMs via Layer-wise Mixture-of-Expertscited by 12
  16. 2024MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editingcited by 9
  17. 2024Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Taskscited by 9
  18. 2024Does Knowledge Localization Hold True? Surprising Differences Between Entity and Relation Perspectives in Language Modelscited by 9
  19. 2025Revealing and Mitigating Over-Attention in Knowledge Editingcited by 8
  20. 2025Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Modelscited by 8
  21. 2024MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Modelscited by 7
  22. 2024MEMLA: Enhancing Multilingual Knowledge Editing with Neuron-Masked Low-Rank Adaptationcited by 6
  23. 2024Lens: Rethinking Multilingual Enhancement for Large Language Modelscited by 6
  24. 2025The Knowledge Microscope: Features as Better Analytical Lenses than Neuronscited by 5
  25. 2025Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Modelscited by 5
  26. 2025Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editingcited by 5
  27. 2026When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Trainingcited by 5
  28. 2025From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformerscited by 4
  29. 2024Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Modelscited by 3
  30. 2024One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Modelscited by 3
  31. 2025Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarkscited by 3
  32. 2026SHIFT: Gate-Modulated Activation Steering for Knowledge Conflict Mitigation in Retrieval-Augmented Generationcited by 3
  33. 2024Neuron Empirical Gradient: Discovering and Quantifying Neurons Global Linear Controllabilitycited by 2
  34. 2024Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoderscited by 1
  35. 2025Understanding Fact Recall in Language Models: Why Two-Stage Training Encourages Memorization but Mixed Training Teaches Knowledgecited by 1
  36. 2025Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Modelscited by 1
  37. 2024FAC2E: Better Understanding Large Language Model Capabilities by Dissociating Language and Cognitioncited by 0
  38. 2025Towards Atoms of Large Language Modelscited by 0

50 most cited of 56 All 56 (JSON)