mlsanityData to 14 Oct 2026

AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models

arXiv 2410.02355 ↗First posted 3 Oct 2024cs.CL cs.AIcited by 236

Not on the map. Why?

Cited by 40 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge. Hence, model editing methods have emerged to enable targeted knowledge updates. To achieve this, a prevailing paradigm is the locating-then-editing approach, which first locates influential parameters and then edits them by introducing a perturbation. While effective, current studies have demonstrated that this perturbation inevitably disrupt the originally preserved knowledge within LLMs, especially in sequential editing scenarios. To address this, we introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters. We theoretically prove that this projection ensures the output of post-edited LLMs remains unchanged when queried about the preserved knowledge, thereby mitigating the issue of disruption. Extensive experiments on various LLMs, including LLaMA3, GPT2-XL, and GPT-J, show that AlphaEdit boosts the performance of most locating-then-editing methods by an average of 36.7% with a single line of additional code for projection solely. Our code is available at: https://github.com/jianghoucheng/AlphaEdit.

Key facts. Cited by 236 papers in the map (data through 14 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

44 of 63 references.

References · 44

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  5. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  6. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  7. 2019Language Models as Knowledge Bases?cited by 1,936
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  9. 2022Mass-Editing Memory in a Transformercited by 1,001
  10. 2018Neural Network Acceptability Judgmentscited by 958
  11. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  12. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
32 more
  1. 2021Editing Factual Knowledge in Language Modelscited by 553
  2. 2021Fast Model Editing at Scalecited by 498
  3. 2022Memory-Based Model Editing at Scalecited by 447
  4. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
  5. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  6. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  7. 2021Gradient Projection Memory for Continual Learningcited by 348
  8. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
  9. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  10. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  11. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  12. 2020Modifying Memories in Transformer Modelscited by 224
  13. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  14. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  15. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  16. 2023PMET: Precise Model Editing in a Transformercited by 173
  17. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  18. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
  19. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  20. 2020Continual Learning in Low-rank Orthogonal Subspacescited by 99
  21. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 93
  22. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Modelscited by 92
  23. 2022TRGP: Trust Region Gradient Projection for Continual Learningcited by 82
  24. 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
  25. 2024The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapsecited by 54
  26. 2024Rebuilding ROME : Resolving Model Collapse during Sequential Model Editingcited by 50
  27. 2024WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editingcited by 40
  28. 2024Knowledge Graph Enhanced Large Language Model Editingcited by 40
  29. 2024Perturbation-Restrained Sequential Model Editingcited by 28
  30. 2024Long-form evaluation of model editingcited by 23
  31. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  32. 2024OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing Systemcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 113; 2026: 116.

236 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
  3. 2025MemOS: A Memory OS for AI Systemcited by 138
  4. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 126
  5. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  6. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 76
  7. 2025MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Modelscited by 73
  8. 2025Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectorscited by 62
  9. 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 53
  10. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 49
  11. 2025LinearRAG: Linear Graph Retrieval Augmented Generation on Large-scale Corporacited by 49
  12. 2025When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generationcited by 48
38 more
  1. 2025Muon Outperforms Adam in Tail-End Associative Memory Learningcited by 46
  2. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 45
  3. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 39
  4. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
  5. 2025SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Modelscited by 36
  6. 2024Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settingscited by 35
  7. 2025Reinforced Lifelong Editing for Language Modelscited by 34
  8. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  9. 2025FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generationcited by 28
  10. 2024Unified Parameter-Efficient Unlearning for LLMscited by 25
  11. 2025You Don't Need Pre-built Graphs for RAG: Retrieval Augmented Generation with Adaptive Reasoning Structurescited by 23
  12. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 21
  13. 2024Neuron-Level Sequential Editing for Large Language Modelscited by 20
  14. 2025Continual Multimodal Contrastive Learningcited by 20
  15. 2024Representation Shattering in Transformers: A Synthetic Study with Knowledge Editingcited by 19
  16. 2024Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergencecited by 19
  17. 2025BiasEdit: Debiasing Stereotyped Language Models via Model Editingcited by 18
  18. 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Modelcited by 16
  19. 2025Knowledge Updating? No More Model Editing! Just Selective Contextual Reasoningcited by 16
  20. 2025SurGE: A Benchmark and Evaluation Framework for Scientific Survey Generationcited by 16
  21. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
  22. 2025Enhancing CLIP Robustness via Cross-Modality Alignmentcited by 15
  23. 2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguardscited by 14
  24. 2025Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?cited by 13
  25. 2025Pre-training Limited Memory Language Models with Internal and External Knowledgecited by 12
  26. 2025Decoupling Reasoning and Knowledge Injection for In-Context Knowledge Editingcited by 12
  27. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 12
  28. 2025Mitigating the Safety Alignment Tax with Null-Space Constrained Policy Optimizationcited by 11
  29. 2025Lifelong Knowledge Editing requires Better Regularizationcited by 10
  30. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  31. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 10
  32. 2024DiffGAD: A Diffusion-based Unsupervised Graph Anomaly Detectorcited by 9
  33. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learnerscited by 9
  34. 2026ST4VLA: Spatially Guided Training for Vision-Language-Action Modelscited by 9
  35. 2025Revealing and Mitigating Over-Attention in Knowledge Editingcited by 8
  36. 2025Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detectioncited by 8
  37. 2025NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detectioncited by 8
  38. 2025Are Robust LLM Fingerprints Adversarially Robust?cited by 8

50 most cited of 236 All 236 (JSON)