# HoReN: Normalized Hopfield Retrieval for Large-Scale Sequential Model Editing

arXiv: https://arxiv.org/abs/2605.08143 · HTML: https://mlsanity.com/paper/2605.08143/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 30 references that are in the map.
- First posted on arXiv 2 May 2026.

## Builds on (most cited first, 28 of 28)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Generalization through Memorization: Nearest Neighbor Language Models](https://mlsanity.com/paper/1911.00172/) (2019)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Hopfield Networks is All You Need](https://mlsanity.com/paper/2008.02217/) (2020)
- [Memory-Based Model Editing at Scale](https://mlsanity.com/paper/2206.06520/) (2022)
- [Zero-Shot Relation Extraction via Reading Comprehension](https://mlsanity.com/paper/1706.04115/) (2017)
- [Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptors](https://mlsanity.com/paper/2211.11031/) (2022)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [A Comprehensive Study of Knowledge Editing for Large Language Models](https://mlsanity.com/paper/2401.01286/) (2024)
- [WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models](https://mlsanity.com/paper/2405.14768/) (2024)
- [WikiBigEdit: Understanding the Limits of Lifelong Knowledge Editing in LLMs](https://mlsanity.com/paper/2503.05683/) (2025)
- [SEA-LION: Southeast Asian Languages in One Network](https://mlsanity.com/paper/2504.05747/) (2025)
- [UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Models](https://mlsanity.com/paper/2505.14679/) (2025)
- [Brain-inspired continual pre-trained learner via silent synaptic consolidation](https://mlsanity.com/paper/2410.05899/) (2024)
- [REPAIR: Robust Editing via Progressive Adaptive Intervention and Reintegration](https://mlsanity.com/paper/2510.01879/) (2025)
- [Distillation-Guided Structural Transfer for Continual Learning Beyond Sparse Distributed Memory](https://mlsanity.com/paper/2512.15267/) (2025)
- [Key-Value Pair-Free Continual Learner via Task-Specific Prompt-Prototype](https://mlsanity.com/paper/2601.04864/) (2026)
- [Reversible Lifelong Model Editing via Semantic Routing-Based LoRA](https://mlsanity.com/paper/2603.11239/) (2026)

## Abstract (arXiv)

Large language models encode vast factual knowledge that can become outdated or incorrect after deployment, yet retraining is prohibitively costly. This motivates lifelong model editing, which updates targeted behavior while preserving the rest of the model. Existing editors, both parameter-modifying and parameter-preserving, degrade severely as edits accumulate and struggle to generalize across paraphrases. We propose HoReN, a codebook-based parameter-preserving editor that wraps a single MLP layer with a discrete key-value memory. HoReN treats each codebook entry as both a knowledge key and a Hopfield stored pattern, retrieves edits by angular similarity on the unit hypersphere, and refines queries through damped Hopfield dynamics so paraphrases converge to the correct memory basin while unrelated inputs remain stable. HoReN achieves strong editing performance with consistent gains across diverse benchmarks spanning standard ZsRE, structured WikiBigEdit, and unstructured UnKE evaluations. Moreover, HoReN scales to 50K sequential edits on ZsRE with stable overall performance above 0.93, while prior editors collapse or degrade severely before reaching 10K. Our code is available at https://github.com/ha11ucin8/HoReN.