mlsanityData to 14 Oct 2026

MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA

arXiv 2312.11795 ↗First posted 19 Dec 2023cs.CLcited by 80

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have shown great success in various Natural Language Processing (NLP) tasks, whist they still need updates after deployment to fix errors or keep pace with the changing knowledge in the world. Researchers formulate such problem as Model Editing and have developed various editors focusing on different axes of editing properties. However, current editors can hardly support all properties and rely on heavy computational resources. In this paper, we propose a plug-in Model Editing method based on neuron-indexed dynamic LoRA (MELO), which alters the behavior of language models by dynamically activating certain LoRA blocks according to the index built in an inner vector database. Our method satisfies various editing properties with high efficiency and can be easily integrated into multiple LLM backbones. Experimental results show that our proposed MELO achieves state-of-the-art editing performance on three sequential editing tasks (document classification, question answering and hallucination correction), while requires the least trainable parameters and computational cost.

Key facts. Cited by 80 papers in the map (data through 14 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

19 of 32 references.

References · 19

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  10. 2022Mass-Editing Memory in a Transformercited by 1,001
  11. 2022Memory-Based Model Editing at Scalecited by 447
  12. 2017Zero-Shot Relation Extraction via Reading Comprehensioncited by 426
7 more
  1. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 372
  2. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 271
  3. 2021SPoT: Better Frozen Model Adaptation through Soft Prompt Transfercited by 235
  4. 2022DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptationcited by 233
  5. 2023Transformer-Patcher: One Mistake worth One Neuroncited by 197
  6. 2022FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processingcited by 49
  7. 2022On Continual Model Refinement in Out-of-Distribution Data Streamscited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 30; 2025: 25; 2026: 25.

80 papers.

Sort:

By area:

  1. 2024AlphaEdit: Null-Space Constrained Knowledge Editing for Language Modelscited by 236
  2. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  3. 2024Detoxifying Large Language Models via Knowledge Editingcited by 87
  4. 2024MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learningcited by 63
  5. 2024Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agentcited by 52
  6. 2024Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learningcited by 40
  7. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  8. 2025Position: Episodic Memory is the Missing Piece for Long-Term LLM Agentscited by 36
  9. 2024In-Context Editing: Learning Knowledge from Self-Induced Distributionscited by 34
  10. 2024Perturbation-Restrained Sequential Model Editingcited by 28
  11. 2024Can Knowledge Editing Really Correct Hallucinations?cited by 28
  12. 2025The Mirage of Model Editing: Revisiting Evaluation in the Wildcited by 27
38 more
  1. 2024Model Editing by Standard Fine-Tuningcited by 23
  2. 2024Editing Conceptual Knowledge for Large Language Modelscited by 21
  3. 2024Neuron-Level Sequential Editing for Large Language Modelscited by 20
  4. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 19
  5. 2024InstructEdit: Instruction-based Knowledge Editing for Large Language Modelscited by 17
  6. 2024EVEDIT: Event-based Knowledge Editing with Deductive Editing Boundariescited by 16
  7. 2024Has this Fact been Edited? Detecting Knowledge Edits in Language Modelscited by 14
  8. 2024Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEditcited by 14
  9. 2025Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?cited by 13
  10. 2024Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Modelscited by 12
  11. 2024Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Modelscited by 10
  12. 2024Editing Factual Knowledge and Explanatory Ability of Medical Large Language Modelscited by 10
  13. 2024MPN: Leveraging Multilingual Patch Neuron for Cross-lingual Model Editingcited by 9
  14. 2025UltraEdit: Training-, Subject-, and Memory-Free Lifelong Editing in Language Modelscited by 9
  15. 2025Fine-tuning Done Right in Model Editingcited by 9
  16. 2024ChroKnowledge: Unveiling Chronological Knowledge of Language Models in Multiple Domainscited by 8
  17. 2025Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editingcited by 8
  18. 2025Exploiting Edited Large Language Models as General Scientific Optimizerscited by 7
  19. 2025BalancEdit: Dynamically Balancing the Generality-Locality Trade-off in Multi-modal Model Editingcited by 7
  20. 2025Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMscited by 7
  21. 2024Towards Incremental Learning in Large Language Models: A Critical Reviewcited by 6
  22. 2025Disentangling Knowledge Representations for Large Language Model Editingcited by 6
  23. 2025MedREK: Retrieval-Based Editing for Medical LLMs with Key-Aware Promptscited by 5
  24. 2026Spectral Characterization and Mitigation of Sequential Knowledge Editing Collapsecited by 5
  25. 2026CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editingcited by 5
  26. 2024Prompt-Guided Foundation Model Tuning for Pathology Image Classificationcited by 4
  27. 2024Composable Interventions for Language Modelscited by 4
  28. 2024Resolving Lexical Bias in Model Editingcited by 4
  29. 2024STACKFEED: Structured Textual Actor-Critic Knowledge Base Editing with FeedBackcited by 4
  30. 2025Temporal Alignment of Time Sensitive Facts with Activation Engineeringcited by 4
  31. 2025SetKE: Knowledge Editing for Knowledge Elements Overlapcited by 3
  32. 2025Memorization and Knowledge Injection in Gated LLMscited by 3
  33. 2025LyapLock: Bounded Knowledge Preservation in Sequential Large Language Model Editingcited by 3
  34. 2025InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editingcited by 2
  35. 2025NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Databasecited by 2
  36. 2025Energy-Regularized Sequential Model Editing on Hyperspherescited by 2
  37. 2026Reverse-Engineering Model Editing on Language Modelscited by 2
  38. 2026FABLE: Fine-grained Fact Anchoring for Unstructured Model Editingcited by 2

50 most cited of 80 All 80 (JSON)