# Model Editing by Standard Fine-Tuning

arXiv: https://arxiv.org/abs/2402.11078 · HTML: https://mlsanity.com/paper/2402.11078/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 12 of 12)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Transformer Feed-Forward Layers Are Key-Value Memories](https://mlsanity.com/paper/2012.14913/) (2020)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [PMET: Precise Model Editing in a Transformer](https://mlsanity.com/paper/2308.08742/) (2023)
- [Editable Neural Networks](https://mlsanity.com/paper/2004.00345/) (2020)
- [MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA](https://mlsanity.com/paper/2312.11795/) (2023)
- [SWEA: Updating Factual Knowledge in Large Language Models via Subject Word Embedding Altering](https://mlsanity.com/paper/2401.17809/) (2024)

## Built on by (most cited first, 23 of 23)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Knowledge Graph Enhanced Large Language Model Editing](https://mlsanity.com/paper/2402.13593/) (2024)
- [Uncovering Overfitting in Large Language Model Editing](https://mlsanity.com/paper/2410.07819/) (2024)
- [CodeUpdateArena: Benchmarking Knowledge Editing on API Updates](https://mlsanity.com/paper/2407.06249/) (2024)
- [Large Scale Knowledge Washing](https://mlsanity.com/paper/2405.16720/) (2024)
- [SAKE: Steering Activations for Knowledge Editing](https://mlsanity.com/paper/2503.01751/) (2025)
- [R3Mem: Bridging Memory Retention and Retrieval via Reversible Compression](https://mlsanity.com/paper/2502.15957/) (2025)
- [Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Models](https://mlsanity.com/paper/2509.24156/) (2025)
- [Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing](https://mlsanity.com/paper/2408.12456/) (2024)
- [Fine-tuning Done Right in Model Editing](https://mlsanity.com/paper/2509.22072/) (2025)
- [Efficiently Quantifying and Mitigating Ripple Effects in Model Editing](https://mlsanity.com/paper/2403.07825/) (2024)
- [GeoEdit: Geometric Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2502.19953/) (2025)
- [Revealing the Deceptiveness of Knowledge Editing: A Mechanistic Analysis of Superficial Editing](https://mlsanity.com/paper/2505.12636/) (2025)
- [Memorization and Knowledge Injection in Gated LLMs](https://mlsanity.com/paper/2504.21239/) (2025)
- [Retention analysis of edited knowledge after fine-tuning](https://mlsanity.com/paper/2507.14198/) (2025)
- [Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model Editing](https://mlsanity.com/paper/2503.11895/) (2025)
- [Towards Effective Model Editing for LLM Personalization](https://mlsanity.com/paper/2512.13676/) (2025)
- [Attesting Model Lineage by Consisted Knowledge Evolution with Fine-Tuning Trajectory](https://mlsanity.com/paper/2601.11683/) (2026)
- [Conflict-Resolving and Sharpness-Aware Minimization for Generalized Knowledge Editing with Multiple Updates](https://mlsanity.com/paper/2602.03696/) (2026)
- [Reinforcement Learning for Neural Model Editing](https://mlsanity.com/paper/2606.13461/) (2026)
- [Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS](https://mlsanity.com/paper/2607.22657/) (2026)
- [KLOD: Locality-Preserving Knowledge Editing via Non-Target Distribution Preservation](https://mlsanity.com/paper/2608.27839/) (2026)
- [Generalizable Model Editing Could Be More Than Direct Generalization](https://mlsanity.com/paper/2610.15707/) (2026)

## Abstract (arXiv)

Standard fine-tuning is considered not as effective as specialized methods for model editing due to its comparatively poor performance. However, it is simple, agnostic to the architectural details of the model being edited, and able to leverage advances in standard training techniques with no additional work (e.g., black-box PEFT for computational efficiency), making it an appealing choice for a model editor. In this work, we show that standard fine-tuning alone can yield competitive model editing performance with two minor modifications. First, we optimize the conditional likelihood rather than the full likelihood. Second, in addition to the typical practice of training on randomly paraphrased edit prompts to encourage generalization, we also train on random or similar unedited facts to encourage locality. Our experiments on the ZsRE and CounterFact datasets demonstrate that these simple modifications allow standard fine-tuning to match or outperform highly specialized editors in terms of edit score.