mlsanityData to 14 Oct 2026

SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing

arXiv 2603.15226 ↗First posted 16 Mar 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

30 of 38 references.

References · 30

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  5. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  6. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  7. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  8. 2022Mass-Editing Memory in a Transformercited by 1,001
  9. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  10. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  11. 2021Fast Model Editing at Scalecited by 498
  12. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
18 more
  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  3. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  4. 2020Modifying Memories in Transformer Modelscited by 224
  5. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  6. 2023Knowledge Editing for Large Language Models: A Surveycited by 194
  7. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  8. 2024Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescuecited by 103
  9. 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 98
  10. 2024Model Editing at Scale leads to Gradual and Catastrophic Forgettingcited by 93
  11. 2023MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRAcited by 80
  12. 2024Locate-then-edit for Multi-hop Factual Recall under Knowledge Editingcited by 37
  13. 2023A Survey on Knowledge Editing of Neural Networkscited by 34
  14. 2025Transcoders Beat Sparse Autoencoders for Interpretabilitycited by 25
  15. 2024Neuron-Level Sequential Editing for Large Language Modelscited by 20
  16. 2025Verifying Chain-of-Thought Reasoning via Its Computational Graphcited by 20
  17. 2024WikiFactDiff: A Large, Realistic, and Temporally Adaptable Dataset for Atomic Factual Knowledge Update in Causal Language Modelscited by 9
  18. 2025REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models
  2. 2026The Anatomy of an Edit: Mechanism-Guided Activation Steering for Knowledge Editing
  3. 2025CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners
  4. 2025Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
  5. 2025Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models
  6. 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
  7. 2026Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
  8. 2026Output Vector Editing for Memorization Mitigation in Large Language Models