# Does Editing Provide Evidence for Localization?

arXiv: https://arxiv.org/abs/2502.11447 · HTML: https://mlsanity.com/paper/2502.11447/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Feb 2025.

## Built on by (most cited first, 7 of 7)

- [The Information Geometry of Softmax: Probing and Steering](https://mlsanity.com/paper/2602.15293/) (2026)
- [Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models](https://mlsanity.com/paper/2505.16252/) (2025)
- [Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networks](https://mlsanity.com/paper/2511.19265/) (2025)
- [On the Effectiveness and Generalization of Race Representations for Debiasing High-Stakes Decisions](https://mlsanity.com/paper/2504.06303/) (2025)
- [Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection](https://mlsanity.com/paper/2602.24021/) (2026)
- [Learning to Weight Parameters for Training Data Attribution](https://mlsanity.com/paper/2506.05647/) (2025)
- [Dynamic Weight Grafting: Localizing Finetuned Factual Knowledge in Transformers](https://mlsanity.com/paper/2506.20746/) (2025)

## Abstract (arXiv)

A basic aspiration for interpretability research in large language models is to "localize" semantically meaningful behaviors to particular components within the LLM. There are various heuristics for finding candidate locations within the LLM. Once a candidate localization is found, it can be assessed by editing the internal representations at the corresponding localization and checking whether this induces model behavior that is consistent with the semantic interpretation of the localization. The question we address here is: how strong is the evidence provided by such edits? To evaluate the localization claim, we want to assess the effect of the optimal intervention at a particular location. The key new technical tool is a way of adapting LLM alignment techniques to find such optimal localized edits. With this tool in hand, we give an example where the edit-based evidence for localization appears strong, but where localization clearly fails. Indeed, we find that optimal edits at random localizations can be as effective as aligning the full model. In aggregate, our results suggest that merely observing that localized edits induce targeted changes in behavior provides little to no evidence that these locations actually encode the target behavior.