# Capability Localization: Capabilities Can be Localized rather than Individual Knowledge

arXiv: https://arxiv.org/abs/2502.20992 · HTML: https://mlsanity.com/paper/2502.20992/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 40 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [Knowledge Neurons in Pretrained Transformers](https://mlsanity.com/paper/2104.08696/) (2021)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [TrustLLM: Trustworthiness in Large Language Models](https://mlsanity.com/paper/2401.05561/) (2024)
- [Practical and Ethical Challenges of Large Language Models in Education: A Systematic Scoping Review](https://mlsanity.com/paper/2303.13379/) (2023)
- [Context Based Emotion Recognition using EMOTIC Dataset](https://mlsanity.com/paper/2003.13401/) (2020)
- [Knowledge Circuits in Pretrained Transformers](https://mlsanity.com/paper/2405.17969/) (2024)
- [The Ethics of ChatGPT in Medicine and Healthcare: A Systematic Review on Large Language Models (LLMs)](https://mlsanity.com/paper/2403.14473/) (2024)
- [NLP for Counterspeech against Hate: A Survey and How-To Guide](https://mlsanity.com/paper/2403.20103/) (2024)
- [EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundaries](https://mlsanity.com/paper/2402.11324/) (2024)
- [Commonsense Knowledge Editing Based on Free-Text in LLMs](https://mlsanity.com/paper/2410.23844/) (2024)
- [Event-level Knowledge Editing](https://mlsanity.com/paper/2402.13093/) (2024)
- [Reasons and Solutions for the Decline in Model Performance after Editing](https://mlsanity.com/paper/2410.23843/) (2024)
- [nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales](https://mlsanity.com/paper/2304.06875/) (2023)
- [SimCLAD: A Simple Framework for Contrastive Learning of Acronym Disambiguation](https://mlsanity.com/paper/2111.14306/) (2021)
- [PSG: Prompt-based Sequence Generation for Acronym Extraction](https://mlsanity.com/paper/2111.14301/) (2021)
- [ADBCMM : Acronym Disambiguation by Building Counterfactuals and Multilingual Mixing](https://mlsanity.com/paper/2112.08991/) (2021)

## Built on by (most cited first, 1 of 1)

- [Know-MRI: A Knowledge Mechanisms Revealer&Interpreter for Large Language Models](https://mlsanity.com/paper/2506.08427/) (2025)

## Abstract (arXiv)

Large scale language models have achieved superior performance in tasks related to natural language processing, however, it is still unclear how model parameters affect performance improvement. Previous studies assumed that individual knowledge is stored in local parameters, and the storage form of individual knowledge is dispersed parameters, parameter layers, or parameter chains, which are not unified. We found through fidelity and reliability evaluation experiments that individual knowledge cannot be localized. Afterwards, we constructed a dataset for decoupling experiments and discovered the potential for localizing data commonalities. To further reveal this phenomenon, this paper proposes a Commonality Neuron Localization (CNL) method, which successfully locates commonality neurons and achieves a neuron overlap rate of 96.42% on the GSM8K dataset. Finally, we have demonstrated through cross data experiments that commonality neurons are a collection of capability neurons that possess the capability to enhance performance. Our code is available at https://github.com/nlpkeg/Capability-Neuron-Localization.