# Cracking Factual Knowledge: A Comprehensive Analysis of Degenerate Knowledge Neurons in Large Language Models

arXiv: https://arxiv.org/abs/2402.13731 · HTML: https://mlsanity.com/paper/2402.13731/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 52 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Time-Aware Language Models as Temporal Knowledge Bases](https://mlsanity.com/paper/2106.15110/) (2021)
- [Physics of Language Models: Part 3.1, Knowledge Storage and Extraction](https://mlsanity.com/paper/2309.14316/) (2023)
- [How Context Affects Language Models' Factual Predictions](https://mlsanity.com/paper/2005.04611/) (2020)
- [An introduction to Topological Data Analysis: fundamental and practical aspects for data scientists](https://mlsanity.com/paper/1710.04019/) (2017)
- [Physics of Language Models: Part 3.2, Knowledge Manipulation](https://mlsanity.com/paper/2309.14402/) (2023)
- [Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2102.00894/) (2021)
- [A roadmap for the computation of persistent homology](https://mlsanity.com/paper/1506.08903/) (2015)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [A Rigorous Study of Integrated Gradients Method and Extensions to Internal Neuron Attributions](https://mlsanity.com/paper/2202.11912/) (2022)
- [Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons](https://mlsanity.com/paper/2308.13198/) (2023)
- [WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing](https://mlsanity.com/paper/2402.10987/) (2024)
- [Approximate Cech Complexes in Low and High Dimensions](https://mlsanity.com/paper/1307.3272/) (2013)

## Built on by (most cited first, 3 of 3)

- [RegMean++: Enhancing Effectiveness and Generalization of Regression Mean for Model Merging](https://mlsanity.com/paper/2508.03121/) (2025)
- [ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domains](https://mlsanity.com/paper/2410.09870/) (2024)
- [One Mind, Many Tongues: A Deep Dive into Language-Agnostic Knowledge Neurons in Large Language Models](https://mlsanity.com/paper/2411.17401/) (2024)

## Abstract (arXiv)

Large language models (LLMs) store extensive factual knowledge, but the underlying mechanisms remain unclear. Previous research suggests that factual knowledge is stored within multi-layer perceptron weights, and some storage units exhibit degeneracy, referred to as Degenerate Knowledge Neurons (DKNs). Despite the novelty and unique properties of this concept, it has not been rigorously defined or systematically studied. We first consider the connection weight patterns of MLP neurons and define DKNs from both structural and functional aspects. Based on this, we introduce the Neurological Topology Clustering method, which allows the formation of DKNs in any numbers and structures, leading to a more accurate DKN acquisition. Furthermore, inspired by cognitive science, we explore the relationship between DKNs and the robustness, evolvability, and complexity of LLMs. Our execution of 34 experiments under 6 settings demonstrates the connection between DKNs and these three properties. The code will be available soon.