mlsanityData to 14 Oct 2026

Weight Patching: Toward Source-Level Mechanistic Localization in LLMs

arXiv 2604.13694 ↗First posted 15 Apr 2026cs.AIcited by 2

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Mechanistic interpretability seeks to localize model behavior to the internal components that causally realize it. Prior work has advanced activation-space localization and causal tracing, but modules that appear important in activation space may merely aggregate or amplify upstream signals rather than encode the target capability in their own parameters. To address this gap, we propose Weight Patching, a parameter-space intervention method for source-oriented analysis in paired same-architecture models that differ in how strongly they express a target capability under the inputs of interest. Given a base model and a behavior-specialized counterpart, Weight Patching replaces selected module weights from the specialized model into the base model under a fixed input. We instantiate the method on instruction following and introduce a framework centered on a vector-anchor behavioral interface that provides a shared internal criterion for whether a task-relevant control state has been formed or recovered in open-ended generation. Under this framework, the analysis reveals a hierarchy from shallow candidate source-side carriers to aggregation and routing modules, and further to downstream execution circuits. The recovered component scores can also guide mechanism-aware model merging, improving selective fusion across the evaluated expert combinations and providing additional external validation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 59 of its 72 references that are in the map. First posted on arXiv 15 Apr 2026.

Builds on

59 of 72 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Mechanistic Interpretability for AI Safety -- A Review22 Apr
  2. 2023Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models7 Nov
  3. 2023Attribution Patching Outperforms Automated Circuit Discovery16 Oct
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretability28 Apr
  5. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability11 Jan
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov

Other references · 53

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2024Qwen2.5 Technical Reportcited by 4,942
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2021Program Synthesis with Large Language Modelscited by 3,637
38 more
  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  6. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  9. 2022Editing Models with Task Arithmeticcited by 1,120
  10. 2023LIMA: Less Is More for Alignmentcited by 1,100
  11. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  12. 2022Mass-Editing Memory in a Transformercited by 1,001
  13. 2022In-context Learning and Induction Headscited by 964
  14. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  15. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  16. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  17. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  18. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 694
  19. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  20. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  21. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  22. 2021Fast Model Editing at Scalecited by 498
  23. 2022Memory-Based Model Editing at Scalecited by 447
  24. 2021Causal Abstractions of Neural Networkscited by 422
  25. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  26. 2023Function Vectors in Large Language Modelscited by 308
  27. 2024How to use and interpret activation patchingcited by 214
  28. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  29. 2023Localizing Model Behavior with Path Patchingcited by 195
  30. 2024Arcee's MergeKit: A Toolkit for Merging Large Language Modelscited by 194
  31. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 160
  32. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 149
  33. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 129
  34. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  35. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 66
  36. 2023From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuningcited by 56
  37. 2024Interpreting and Improving Large Language Models in Arithmetic Calculationcited by 45
  38. 2025Activation-Informed Merging of Large Language Modelscited by 30

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flowcited by 0
  2. 2026Beyond Endpoint Gains: A Weight-Delta Audit of Medical Specializationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Navigating by Old Maps: The Pitfalls of Static Mechanistic Localization in LLM Post-Training
  2. 2026Local Linearity of LLMs Enables Activation Steering via Model-Based Linear Optimal Control
  3. 2026Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning
  4. 2026Predictive Memory Localization: Forecasting Selective Intervention Paths from Internal Signals
  5. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safety
  6. 2025Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagation
  7. 2025Findings of the BlackboxNLP 2025 Shared Task: Localizing Circuits and Causal Variables in Language Models
  8. 2025Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs