mlsanityData to 7 Oct 2026

Task-Specific Skill Localization in Fine-tuned Language Models

arXiv 2302.06600 ↗First posted 13 Feb 2023cs.CL cs.LGcited by 104

Not on the map. Why?

Abstract (arXiv)

Pre-trained language models can be fine-tuned to solve diverse NLP tasks, including in few-shot settings. Thus fine-tuning allows the model to quickly pick up task-specific ``skills,'' but there has been limited study of where these newly-learnt skills reside inside the massive model. This paper introduces the term skill localization for this problem and proposes a solution. Given the downstream task and a model fine-tuned on that task, a simple optimization is used to identify a very small subset of parameters (~0.01% of model parameters) responsible for (>95%) of the model's performance, in the sense that grafting the fine-tuned values for just this tiny subset onto the pre-trained model gives performance almost as well as the fine-tuned model. While reminiscent of recent works on parameter-efficient fine-tuning, the novel aspects here are that: (i) No further re-training is needed on the subset (unlike, say, with lottery tickets). (ii) Notable improvements are seen over vanilla fine-tuning with respect to calibration of predictions in-distribution (40-90% error reduction) as well as the quality of predictions out-of-distribution (OOD). In models trained on multiple tasks, a stronger notion of skill localization is observed, where the sparse regions corresponding to different tasks are almost disjoint, and their overlap (when it happens) is a proxy for task similarity. Experiments suggest that localization via grafting can assist certain forms of continual learning.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). Builds on 45 of its 61 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

45 of 61 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  12. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
33 more
  1. 2021Robust fine-tuning of zero-shot modelscited by 785
  2. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  3. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  4. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  5. 2016Cross-stitch Networks for Multi-task Learningcited by 466
  6. 2019Multi-Task Deep Neural Networks for Natural Language Understandingcited by 349
  7. 2019Verified Uncertainty Calibrationcited by 307
  8. 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
  9. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  10. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  11. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  12. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  13. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  14. 2021Training Neural Networks with Fixed Sparse Maskscited by 212
  15. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  16. 2019SGD on Neural Networks Learns Functions of Increasing Complexitycited by 203
  17. 2021Revisiting Model Stitching to Compare Neural Representationscited by 203
  18. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 184
  19. 2016Recurrent Neural Network for Text Classification with Multi-Task Learningcited by 144
  20. 2022A Kernel-Based View of Language Model Fine-Tuningcited by 118
  21. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  22. 2016Fully-adaptive Feature Sharing in Multi-Task Networks with Applications in Person Attribute Classificationcited by 93
  23. 2021Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcited by 82
  24. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  25. 2020A Mathematical Exploration of Why Language Models Help Solve Downstream Taskscited by 71
  26. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 65
  27. 2021A Winning Hand: Compressing Deep Networks Can Improve Out-Of-Distribution Robustnesscited by 62
  28. 2021Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalizationcited by 56
  29. 2019In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictorscited by 44
  30. 2022How to Fine-Tune Vision Models with SGDcited by 23
  31. 2022Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learningcited by 13
  32. 2021Adapting by Pruning: A Case Study on BERTcited by 11
  33. 2022A Win-win Deal: Towards Sparse and Robust Pre-trained Language Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 44; 2025: 22; 2026: 23.

104 papers.

Sort:

By area:

  1. 2023Function Vectors in Large Language Modelscited by 292
  2. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  3. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  4. 2023Mitigating the Alignment Tax of RLHFcited by 181
  5. 2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 140
  6. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  7. 2024Localizing Task Information for Improved Model Merging and Compressioncited by 130
  8. 2024Real-Time Anomaly Detection and Reactive Planning with Large Language Modelscited by 67
  9. 2024LoFiT: Localized Fine-tuning on LLM Representationscited by 56
  10. 2024Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Modelscited by 54
  11. 2024Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmeticcited by 50
  12. 2023Continual Instruction Tuning for Large Multimodal Modelscited by 49
38 more
  1. 2025Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelscited by 46
  2. 2024Interpreting and Improving Large Language Models in Arithmetic Calculationcited by 43
  3. 2023When Do Prompting and Prefix-Tuning Work? A Theory of Capabilities and Limitationscited by 41
  4. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  5. 2024The LLM Language Network: A Neuroscientific Approach for Identifying Causally Task-Relevant Unitscited by 34
  6. 2024Towards Understanding the Fragility of Multilingual LLMs against Fine-Tuning Attackscited by 33
  7. 2023A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Timecited by 30
  8. 2025MergeBench: A Benchmark for Merging Domain-Specialized LLMscited by 30
  9. 2023Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarkscited by 29
  10. 2024APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inferencecited by 29
  11. 2024Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsitycited by 27
  12. 2023Mitigating Biases for Instruction-following Language Models via Bias Neurons Eliminationcited by 23
  13. 2024Configurable Foundation Models: Building LLMs from a Modular Perspectivecited by 22
  14. 2024Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Mergingcited by 22
  15. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  16. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  17. 2024Understanding Layer Significance in LLM Alignmentcited by 19
  18. 2024Separate the Wheat from the Chaff: A Post-Hoc Approach to Safety Re-Alignment for Fine-Tuned Language Modelscited by 19
  19. 2024TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidationcited by 18
  20. 2025Representation Engineering for Large-Language Models: Survey and Research Challengescited by 18
  21. 2025Continual Learning via Sparse Memory Finetuningcited by 17
  22. 2023Why Softmax Attention Outperforms Linear Attentioncited by 15
  23. 2024In-Context Learning State Vector with Inner and Momentum Optimizationcited by 15
  24. 2024RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Datacited by 15
  25. 2025Recurrent Knowledge Identification and Fusion for Language Model Continual Learningcited by 14
  26. 2024Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinationscited by 13
  27. 2024MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuningcited by 13
  28. 2023Uncovering Intermediate Variables in Transformers using Circuit Probingcited by 12
  29. 2024HiFT: A Hierarchical Full Parameter Fine-Tuning Strategycited by 12
  30. 2024SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgerycited by 12
  31. 2025Mitigating Catastrophic Forgetting in Large Language Models with Forgetting-aware Pruningcited by 12
  32. 2023Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weightscited by 11
  33. 2024Parameter-Efficient Fine-Tuning of State Space Modelscited by 11
  34. 2025Learning Task Representations from In-Context Learningcited by 11
  35. 2025Neural Parameter Search for Slimmer Fine-Tuned Models and Better Transfercited by 11
  36. 2024Understanding Synthetic Context Extension via Retrieval Headscited by 10
  37. 2023NeuroSurgeon: A Toolkit for Subnetwork Analysiscited by 9
  38. 2023From Channel Bias to Feature Redundancy: Uncovering the "Less is More" Principle in Few-Shot Learningcited by 9

50 most cited of 104 All 104 (JSON)