# One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them

arXiv: https://arxiv.org/abs/2605.28839 · HTML: https://mlsanity.com/paper/2605.28839/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 38 references that are in the map.
- First posted on arXiv 18 Apr 2026.

## Builds on (most cited first, 29 of 29)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models](https://mlsanity.com/paper/2301.04213/) (2023)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [An empirical study of LLaMA3 quantization: from LLMs to MLLMs](https://mlsanity.com/paper/2404.14047/) (2024)
- [The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse](https://mlsanity.com/paper/2402.09656/) (2024)
- [A Survey on Knowledge Editing of Neural Networks](https://mlsanity.com/paper/2310.19704/) (2023)
- [Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models](https://mlsanity.com/paper/2310.16570/) (2023)
- [Why Does New Knowledge Create Messy Ripple Effects in LLMs?](https://mlsanity.com/paper/2407.12828/) (2024)
- [Evaluating Dependencies in Fact Editing for Language Models: Specificity and Implication Awareness](https://mlsanity.com/paper/2312.01858/) (2023)
- [Has this Fact been Edited? Detecting Knowledge Edits in Language Models](https://mlsanity.com/paper/2405.02765/) (2024)
- [Revealing and Mitigating Over-Attention in Knowledge Editing](https://mlsanity.com/paper/2502.14838/) (2025)
- [The Queen of England is not England's Queen: On the Lack of Factual Coherency in PLMs](https://mlsanity.com/paper/2402.01453/) (2024)
- [BalancEdit: Dynamically Balancing the Generality-Locality Trade-off in Multi-modal Model Editing](https://mlsanity.com/paper/2505.01343/) (2025)
- [Tracing and Reversing Edits in LLMs](https://mlsanity.com/paper/2505.20819/) (2025)
- [Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editing](https://mlsanity.com/paper/2505.12636/) (2025)
- [Knowledge Editing for Large Language Model with Knowledge Neuronal Ensemble](https://mlsanity.com/paper/2412.20637/) (2024)
- [Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation](https://mlsanity.com/paper/2511.05852/) (2025)
- [Persuasion Tokens for Editing Factual Knowledge in LLMs](https://mlsanity.com/paper/2601.16781/) (2026)

## Built on by (most cited first, 2 of 2)

- [Exposing the Illusion of Erasure in Knowledge Editing for LLMs](https://mlsanity.com/paper/2606.23276/) (2026)
- [Can a Language Model Learn Facts Continually in Its Weights?](https://mlsanity.com/paper/2607.11020/) (2026)

## Abstract (arXiv)

Knowledge editing methods such as ROME and MEMIT update factual associations in transformer models by modifying MLP weights. While evaluated mainly by output behavior, their internal mechanism remains underexplored. We investigate whether edits rely on a common mechanism, regardless of which fact is modified. Despite fact-specific weight changes, we argue that ROME and MEMIT target the same subset of weights critical for maintaining edits. To isolate this subset, we train a compact binary mask over the edited weights. The mask reverses 80% of edits on the training set and over 70% on the test set, confirming that diverse edits share a common functional structure. Our analysis reveals that the mask reverses edits by eliminating overattention in later layers. Additionally, we show that injecting the mask during editing drops editing success from 98% to 38%, demonstrating that this mechanism is necessary for edits to succeed. Our finding that edits suppress rather than overwrite knowledge explains why ROME and MEMIT fail to propagate changes to related facts. The identified common functional subspace informs detection and defense against unwanted edits.