mlsanityData to 7 Oct 2026

LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples

arXiv 2512.07375 ↗First posted 8 Dec 2025cs.LG cs.CLcited by 8

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large language models (LLMs) possess vast knowledge acquired from extensive training corpora, but they often cannot remove specific pieces of information when needed, which makes it hard to handle privacy, bias mitigation, and knowledge correction. Traditional model unlearning approaches require computationally expensive fine-tuning or direct weight editing, making them impractical for real-world deployment. In this work, we introduce LoRA-based Unlearning with Negative Examples (LUNE), a lightweight framework that performs negative-only unlearning by updating only low-rank adapters while freezing the backbone, thereby localizing edits and avoiding disruptive global changes. Leveraging Low-Rank Adaptation (LoRA), LUNE targets intermediate representations to suppress (or replace) requested knowledge with an order-of-magnitude lower compute and memory than full fine-tuning or direct weight editing. Extensive experiments on multiple factual unlearning tasks show that LUNE: (I) achieves effectiveness comparable to full fine-tuning and memory-editing methods, and (II) reduces computational cost by about an order of magnitude.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 48 of its 63 references that are in the map. First posted on arXiv 8 Dec 2025.

Builds on

48 of 63 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Scalable and Robust LLM Unlearning by Correcting Responses with Retrieved Exclusions30 Sep
  2. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond26 Feb
  3. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models22 Feb
  4. 2024Evaluating Deep Unlearning in Large Language Models19 Oct
  5. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  6. 2024Dissecting Fine-Tuning Unlearning in Large Language Models9 Oct
  7. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMs13 Aug
  8. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models16 Jun
  9. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  10. 2024Machine Unlearning of Pre-trained Large Language Models23 Feb
  11. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  12. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
3 more
  1. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  2. 2023Large Language Model Unlearning14 Oct
  3. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct

Other references · 33

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  11. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  12. 2022Editing Models with Task Arithmeticcited by 1,076
21 more
  1. 2019Machine Unlearningcited by 1,020
  2. 2022Mass-Editing Memory in a Transformercited by 976
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  4. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 780
  5. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  6. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 740
  7. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 686
  8. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  9. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  10. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  11. 2021Compacter: Efficient Low-Rank Hypercomplex Adapter Layerscited by 458
  12. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  13. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 390
  14. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  15. 2020Parameter-Efficient Transfer Learning with Diff Pruningcited by 351
  16. 2020AdapterDrop: On the Efficiency of Adapters in Transformerscited by 223
  17. 2024Promoting Fairness in Link Prediction with Graph Enhancementcited by 6
  18. 2025Beyond Chunks and Graphs: Retrieval-Augmented Generation through Triplet-Driven Thinkingcited by 6
  19. 2025Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?cited by 6
  20. 2024FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devicescited by 5
  21. 2025Enabling Group Fairness in Graph Unlearning via Bi-level Debiasingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agentscited by 7
  2. 2026Position: The Term "Machine Unlearning" Is Overused in LLMscited by 3
  3. 2026Anatomy of Unlearning: The Dual Impact of Fact Salience and Model Fine-Tuningcited by 1
  4. 2026MEMOREPAIR: Barrier-First Cascade Repair in Agentic Memorycited by 1
  5. 2026Shadow Unlearning: A Neuro-Semantic Approach to Fidelity-Preserving Faceless Forgetting in LLMscited by 0
  6. 2026Interference-Aware Multi-Task Unlearningcited by 0
  7. 2026Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearningcited by 0
  8. 2026Can Scientific Claims Be Removed from Large Language Models? A Systematic Evaluation of Claim-Level Unlearningcited by 0