When to Write and When to Suppress: Route-Specialized Dual Adapters for Memory-Assisted Knowledge Editing
Not on the map. Why?
Abstract (arXiv)
Knowledge editing systems must update selected facts while preserving nearby but irrelevant behavior. This paper studies this problem in a memory-assisted setting where an edit memory is retrieved at inference time and a parameter-efficient adapter corrects the model's object preference. We argue that the central design question is not only how to write an edit, but also when to suppress it. We introduce RRDA, a route-specialized dual-adapter editor. A relevance router first decides whether a prompt should receive an edit memory. Routed prompts use an edit adapter trained to prefer the new object over the original object; unrouted non-direct prompts use a separate locality adapter trained to preserve or restore the original-object preference. We evaluate RRDA on three 1,000-case protocols, CounterFact, ZsRE, and MQuAKE-CF, under the same memory protocol and two 7B/8B base models. On Llama-3.1-8B-Instruct, RRDA obtains the best overall probability-preference accuracy on all three benchmarks: 0.8180 on CounterFact, 0.8946 on ZsRE, and 0.9922 on MQuAKE-CF. The same trend holds on Qwen3-8B. Router ablations show that the relevant memory boundary differs across datasets: a lexical neural router is safest on CounterFact, while BGE embedding routing is better on ZsRE and MQuAKE-CF. Memory, component, and module ablations show that explicit memory supplies the largest edit gain, while route-specialized adapters improve the final reliability-locality balance rather than simply increasing LoRA capacity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 30 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
26 of 30 references.
References · 26
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2025Qwen3 Technical Reportcited by 8,820
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,463
- 2022Mass-Editing Memory in a Transformercited by 1,001
- 2021Editing Factual Knowledge in Language Modelscited by 553
- 2021Fast Model Editing at Scalecited by 498
- 2022Memory-Based Model Editing at Scalecited by 447
- 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
- 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 300
14 more
- 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
- 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
- 2023Evaluating the Ripple Effects of Knowledge Editing in Language Modelscited by 242
- 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
- 2020Modifying Memories in Transformer Modelscited by 224
- 2023EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Modelscited by 98
- 2024WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Modelscited by 92
- 2023Massive Editing for Large Language Models via Meta Learningcited by 63
- 2024Perturbation-Restrained Sequential Model Editingcited by 28
- 2024RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuningcited by 21
- 2024Time Sensitive Knowledge Editing through Efficient Finetuningcited by 14
- 2025Fine-tuning Done Right in Model Editingcited by 9
- 2025MemEIC: A Step Toward Continual and Compositional Knowledge Editingcited by 2
- 2025Norm Growth and Stability Challenges in Localized Sequential Knowledge Editingcited by 1
Built on by
Not cited yet.