# Efficient Knowledge Editing via Minimal Precomputation

arXiv: https://arxiv.org/abs/2506.04226 · HTML: https://mlsanity.com/paper/2506.04226/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 17 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 14 of 14)

- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [Model Editing at Scale leads to Gradual and Catastrophic Forgetting](https://mlsanity.com/paper/2401.07453/) (2024)
- [A Unified Framework for Model Editing](https://mlsanity.com/paper/2403.14236/) (2024)
- [Rebuilding ROME : Resolving Model Collapse during Sequential Model Editing](https://mlsanity.com/paper/2403.07175/) (2024)
- [Lifelong Knowledge Editing requires Better Regularization](https://mlsanity.com/paper/2502.01636/) (2025)
- [Is Bigger Edit Batch Size Always Better? -- An Empirical Study on Model Editing with Llama-3](https://mlsanity.com/paper/2405.00664/) (2024)

## Built on by (most cited first, 3 of 3)

- [EtCon: Edit-then-Consolidate for Reliable Knowledge Editing](https://mlsanity.com/paper/2512.04753/) (2025)
- [Beyond the Covariance Trap: Unlocking Generalization in Same-Subject Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2603.15518/) (2026)
- [BetaEdit: Null-Space Constrained Sequential Model Editing](https://mlsanity.com/paper/2605.09285/) (2026)

## Abstract (arXiv)

Knowledge editing methods like MEMIT are able to make data and compute efficient updates of factual knowledge by using a single sentence to update facts and their consequences. However, what is often overlooked is a "precomputation step", which requires a one-time but significant computational cost. The authors of MEMIT originally precompute approximately 44 million hidden vectors per edited layer, which requires a forward pass over 44 million tokens. For GPT-J (6B), this precomputation step takes 36 hours on a single GPU, while it takes approximately 40 hours for Llama2-7B. Additionally, this precomputation time grows with model size. In this paper, we show that this excessive computational cost is unnecessary. Knowledge editing using MEMIT and related methods, such as ROME and EMMET, can be performed by pre-computing a very small portion of the 44 million hidden vectors. We first present the theoretical minimum number of hidden vector precomputation required for solutions of these editing methods to exist. We then empirically show that knowledge editing using these methods can be done by pre-computing significantly fewer hidden vectors. Specifically, we show that the precomputation step can be done with less than 0.3% of the originally stipulated number of hidden vectors. This saves a significant amount of precomputation time and allows users to begin editing new models within a few minutes.