# GeoEdit: Geometric Knowledge Editing for Large Language Models

arXiv: https://arxiv.org/abs/2502.19953 · HTML: https://mlsanity.com/paper/2502.19953/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 28 of 28)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Editing Factual Knowledge in Language Models](https://mlsanity.com/paper/2104.08164/) (2021)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Editing Large Language Models: Problems, Methods, and Opportunities](https://mlsanity.com/paper/2305.13172/) (2023)
- [Can We Edit Factual Knowledge by In-Context Learning?](https://mlsanity.com/paper/2305.12740/) (2023)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue](https://mlsanity.com/paper/2401.04700/) (2024)
- [Detoxifying Large Language Models via Knowledge Editing](https://mlsanity.com/paper/2403.14472/) (2024)
- [A Unified Framework for Model Editing](https://mlsanity.com/paper/2403.14236/) (2024)
- [Learning to Edit: Aligning LLMs with Knowledge Editing](https://mlsanity.com/paper/2402.11905/) (2024)
- [Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts](https://mlsanity.com/paper/2405.11613/) (2024)
- [Model Editing by Standard Fine-Tuning](https://mlsanity.com/paper/2402.11078/) (2024)
- [The Law of Knowledge Overshadowing: Towards Understanding, Predicting, and Preventing LLM Hallucination](https://mlsanity.com/paper/2502.16143/) (2025)
- [Understanding Layer Significance in LLM Alignment](https://mlsanity.com/paper/2410.17875/) (2024)
- [TaSL: Continual Dialog State Tracking via Task Skill Localization and Consolidation](https://mlsanity.com/paper/2408.09857/) (2024)
- [Commonsense Knowledge Editing Based on Free-Text in LLMs](https://mlsanity.com/paper/2410.23844/) (2024)
- [Recurrent Knowledge Identification and Fusion for Language Model Continual Learning](https://mlsanity.com/paper/2502.17510/) (2025)
- [Adaptive Token Biaser: Knowledge Editing via Biasing Key Entities](https://mlsanity.com/paper/2406.12468/) (2024)
- [Parenting: Optimizing Knowledge Selection of Retrieval-Augmented Language Models with Parameter Decoupling and Tailored Tuning](https://mlsanity.com/paper/2410.10360/) (2024)
- [Consecutive Batch Model Editing with HooK Layers](https://mlsanity.com/paper/2403.05330/) (2024)
- [Low-Rank Interconnected Adaptation across Layers](https://mlsanity.com/paper/2407.09946/) (2024)
- [KIF: Knowledge Identification and Fusion for Language Model Continual Learning](https://mlsanity.com/paper/2408.05200/) (2024)
- [Continual Dialogue State Tracking via Reason-of-Select Distillation](https://mlsanity.com/paper/2408.09846/) (2024)
- [RIPPLECOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learning](https://mlsanity.com/paper/2410.03122/) (2024)
- [Better Call SAUL: Fluent and Consistent Language Model Editing with Generation Regularization](https://mlsanity.com/paper/2410.02433/) (2024)

## Built on by (most cited first, 6 of 6)

- [When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation](https://mlsanity.com/paper/2506.05690/) (2025)
- [Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning](https://mlsanity.com/paper/2509.18942/) (2025)
- [Geometric-disentangelment Unlearning](https://mlsanity.com/paper/2511.17100/) (2025)
- [Mitigating Memorization Where It Happens](https://mlsanity.com/paper/2602.08901/) (2026)
- [Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Models](https://mlsanity.com/paper/2605.28828/) (2026)
- [Output Vector Editing for Memorization Mitigation in Large Language Models](https://mlsanity.com/paper/2606.18767/) (2026)

## Abstract (arXiv)

Regular updates are essential for maintaining up-to-date knowledge in large language models (LLMs). Consequently, various model editing methods have been developed to update specific knowledge within LLMs. However, training-based approaches often struggle to effectively incorporate new knowledge while preserving unrelated general knowledge. To address this challenge, we propose a novel framework called Geometric Knowledge Editing (GeoEdit). GeoEdit utilizes the geometric relationships of parameter updates from fine-tuning to differentiate between neurons associated with new knowledge updates and those related to general knowledge perturbations. By employing a direction-aware knowledge identification method, we avoid updating neurons with directions approximately orthogonal to existing knowledge, thus preserving the model's generalization ability. For the remaining neurons, we integrate both old and new knowledge for aligned directions and apply a "forget-then-learn" editing strategy for opposite directions. Additionally, we introduce an importance-guided task vector fusion technique that filters out redundant information and provides adaptive neuron-level weighting, further enhancing model editing performance. Extensive experiments on two publicly available datasets demonstrate the superiority of GeoEdit over existing state-of-the-art methods.