# Unveiling Language Competence Neurons: A Psycholinguistic Approach to Model Interpretability

arXiv: https://arxiv.org/abs/2409.15827 · HTML: https://mlsanity.com/paper/2409.15827/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 24 Sep 2024.

## Builds on (most cited first, 16 of 16)

- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Understanding the Role of Individual Units in a Deep Neural Network](https://mlsanity.com/paper/2009.05041/) (2020)
- [Compositional Explanations of Neurons](https://mlsanity.com/paper/2006.14032/) (2020)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models](https://mlsanity.com/paper/2402.16438/) (2024)
- [Open Sesame: Getting Inside BERT's Linguistic Knowledge](https://mlsanity.com/paper/1906.01698/) (2019)
- [Identifying Semantic Induction Heads to Understand In-Context Learning](https://mlsanity.com/paper/2402.13055/) (2024)
- [Do large language models resemble humans in language use?](https://mlsanity.com/paper/2303.08014/) (2023)
- [Sharing Matters: Analysing Neurons Across Languages and Tasks in LLMs](https://mlsanity.com/paper/2406.09265/) (2024)
- [Evaluating Recurrent Neural Network Explanations](https://mlsanity.com/paper/1904.11829/) (2019)
- [Understanding Addition and Subtraction in Transformers](https://mlsanity.com/paper/2402.02619/) (2024)
- [Emergent inabilities? Inverse scaling over the course of pretraining](https://mlsanity.com/paper/2305.14681/) (2023)

## Built on by (most cited first, 8 of 8)

- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [Language-specific Neurons Do Not Facilitate Cross-Lingual Transfer](https://mlsanity.com/paper/2503.17456/) (2025)
- [How Syntax Specialization Emerges in Language Models](https://mlsanity.com/paper/2505.19548/) (2025)
- [A Review of Incorporating Psychological Theories in LLMs](https://mlsanity.com/paper/2505.00003/) (2025)
- [Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models](https://mlsanity.com/paper/2502.20408/) (2025)
- [From Monolingual to Bilingual: Investigating Language Conditioning in Large Language Models for Psycholinguistic Tasks](https://mlsanity.com/paper/2508.02502/) (2025)
- [Knowing Bias, Doing Better: Mitigating Social Bias in LLMs via Know-Bias Neuron Enhancement](https://mlsanity.com/paper/2601.21864/) (2026)
- [GKnow: Measuring the Entanglement of Gender Bias and Factual Gender](https://mlsanity.com/paper/2605.12299/) (2026)

## Abstract (arXiv)

As large language models (LLMs) advance in their linguistic capacity, understanding how they capture aspects of language competence remains a significant challenge. This study therefore employs psycholinguistic paradigms in English, which are well-suited for probing deeper cognitive aspects of language processing, to explore neuron-level representations in language model across three tasks: sound-shape association, sound-gender association, and implicit causality. Our findings indicate that while GPT-2-XL struggles with the sound-shape task, it demonstrates human-like abilities in both sound-gender association and implicit causality. Targeted neuron ablation and activation manipulation reveal a crucial relationship: When GPT-2-XL displays a linguistic ability, specific neurons correspond to that competence; conversely, the absence of such an ability indicates a lack of specialized neurons. This study is the first to utilize psycholinguistic experiments to investigate deep language competence at the neuron level, providing a new level of granularity in model interpretability and insights into the internal mechanisms driving language ability in the transformer-based LLM.