# Let's Focus on Neuron: Neuron-Level Supervised Fine-tuning for Large Language Model

arXiv: https://arxiv.org/abs/2403.11621 · HTML: https://mlsanity.com/paper/2403.11621/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 38 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 24 of 24)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://mlsanity.com/paper/2301.00774/) (2023)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning](https://mlsanity.com/paper/2205.05638/) (2022)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Parameter-Efficient Transfer Learning with Diff Pruning](https://mlsanity.com/paper/2012.07463/) (2020)
- [The Lottery Ticket Hypothesis for Pre-trained BERT Networks](https://mlsanity.com/paper/2007.12223/) (2020)
- [Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time](https://mlsanity.com/paper/2310.17157/) (2023)
- [AdapterDrop: On the Efficiency of Adapters in Transformers](https://mlsanity.com/paper/2010.11918/) (2020)
- [PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU](https://mlsanity.com/paper/2312.12456/) (2023)
- [Training Neural Networks with Fixed Sparse Masks](https://mlsanity.com/paper/2111.09839/) (2021)
- [Composable Sparse Fine-Tuning for Cross-Lingual Transfer](https://mlsanity.com/paper/2110.07560/) (2021)
- [When BERT Plays the Lottery, All Tickets Are Winning](https://mlsanity.com/paper/2005.00561/) (2020)
- [Monarch: Expressive Structured Matrices for Efficient and Accurate Training](https://mlsanity.com/paper/2204.00595/) (2022)
- [Analyzing Individual Neurons in Pre-trained Language Models](https://mlsanity.com/paper/2010.02695/) (2020)
- [Neurons in Large Language Models: Dead, N-gram, Positional](https://mlsanity.com/paper/2309.04827/) (2023)
- [CrossSum: Beyond English-Centric Cross-Lingual Summarization for 1,500+ Language Pairs](https://mlsanity.com/paper/2112.08804/) (2021)
- [Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation](https://mlsanity.com/paper/2212.10551/) (2022)

## Built on by (most cited first, 8 of 8)

- [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026)
- [RepreGuard: Detecting LLM-Generated Text by Revealing Hidden Representation Patterns](https://mlsanity.com/paper/2508.13152/) (2025)
- [More Experts Than Galaxies: Conditionally-overlapping Experts With Biologically-Inspired Fixed Routing](https://mlsanity.com/paper/2410.08003/) (2024)
- [Sparse Subnetwork Enhancement for Underrepresented Languages in Large Language Models](https://mlsanity.com/paper/2510.13580/) (2025)
- [Neuron-Aware Data Selection In Instruction Tuning For Large Language Models](https://mlsanity.com/paper/2603.13201/) (2026)
- [Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning](https://mlsanity.com/paper/2607.09287/) (2026)
- [Neuron-level Balance between Stability and Plasticity in Deep Reinforcement Learning](https://mlsanity.com/paper/2504.08000/) (2025)
- [Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spaces](https://mlsanity.com/paper/2505.07831/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are composed of neurons that exhibit various behaviors and roles, which become increasingly diversified as models scale. Recent studies have revealed that not all neurons are active across different datasets, and this sparsity correlates positively with the task-specific ability, leading to advancements in model pruning and training efficiency. Traditional fine-tuning methods engage all parameters of LLMs, which is computationally expensive and may not be necessary. In contrast, Parameter-Efficient Fine-Tuning (PEFT) approaches aim to minimize the number of trainable parameters, yet they still operate at a relatively macro scale (e.g., layer-level). We introduce Neuron-Level Fine-Tuning (NeFT), a novel approach that refines the granularity of parameter training down to the individual neuron, enabling more precise and computationally efficient model updates. The experimental results show that NeFT not only exceeded the performance of full-parameter fine-tuning and PEFT but also provided insights into the analysis of neurons.