mlsanityData to 14 Oct 2026

Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning

arXiv 2410.07163 ↗First posted 9 Oct 2024cs.CL cs.AI cs.LGcited by 172

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

This work studies the problem of large language model (LLM) unlearning, aiming to remove unwanted data influences (e.g., copyrighted or harmful content) while preserving model utility. Despite the increasing demand for unlearning, a technically-grounded optimization framework is lacking. Gradient ascent (GA)-type methods, though widely used, are suboptimal as they reverse the learning process without controlling optimization divergence (i.e., deviation from the pre-trained state), leading to risks of over-forgetting and potential model collapse. Negative preference optimization (NPO) has been proposed to address this issue and is considered one of the state-of-the-art LLM unlearning approaches. In this work, we revisit NPO and identify another critical issue: reference model bias. This bias arises from using the reference model (i.e., the model prior to unlearning) to evaluate the unlearning success, which can compromise NPO's effectiveness. Specifically, it leads to (a) uneven allocation of optimization power across forget data with varying difficulty levels and (b) ineffective gradient weight smoothing during the early stages of unlearning optimization. To overcome these challenges, we propose a simple yet effective unlearning optimization framework, called SimNPO, showing that `simplicity' in removing the reliance on a reference model (through the lens of simple preference optimization) benefits unlearning. We provide deeper insights into SimNPO's advantages through an analysis based on mixtures of Markov chains. Extensive experiments further validate SimNPO's efficacy on benchmarks like TOFU and MUSE, as well as its robustness against relearning attacks. Codes are available at https://github.com/OPTML-Group/Unlearn-Simple.

Key facts. Cited by 172 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 37 of its 67 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

37 of 67 references.

References · 37

  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  2. 2017Decoupled Weight Decay Regularizationcited by 1,831
  3. 2022Editing Models with Task Arithmeticcited by 1,120
  4. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  5. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  6. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
  7. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 437
  8. 2023SLiC-HF: Sequence Likelihood Calibration with Human Feedbackcited by 401
  9. 2022A Survey of Machine Unlearningcited by 312
  10. 2023Large Language Model Unlearningcited by 302
  11. 2023Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Modelscited by 301
  12. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
25 more
  1. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 261
  2. 2021Machine Unlearning of Features and Labelscited by 229
  3. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 229
  4. 2023In-Context Unlearning: Language Models as Few Shot Unlearnerscited by 224
  5. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 163
  6. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 116
  7. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  8. 2024Guardrail Baselines for Unlearning in LLMscited by 96
  9. 2022Federated Unlearning: How to Efficiently Erase a Client in FL?cited by 95
  10. 2024Rethinking LLM Memorization through the Lens of Adversarial Compressioncited by 94
  11. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 84
  12. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 79
  13. 2024UnlearnCanvas: Stylized Image Dataset for Enhanced Machine Unlearning Evaluation in Diffusion Modelscited by 50
  14. 2024Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitioncited by 50
  15. 2022Certified Graph Unlearningcited by 46
  16. 2024Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearningcited by 44
  17. 2023Efficient RLHF: Reducing the Memory Usage of PPOcited by 40
  18. 2023Knowledge Sanitization of Large Language Modelscited by 37
  19. 2022Privacy Adhering Machine Un-learning in NLPcited by 36
  20. 2024Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Modelscited by 33
  21. 2024RKLD: Reverse KL-Divergence-based Knowledge Distillation for Unlearning Personal Information in Large Language Modelscited by 21
  22. 2024Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMscited by 18
  23. 2024MEOW: MEMOry Supervised LLM Unlearning Via Inverted Factscited by 18
  24. 2024Large Scale Knowledge Washingcited by 16
  25. 2023Forgettable Federated Linear Learning with Certified Data Unlearningcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 66; 2026: 106.

172 papers.

Sort:

By area:

  1. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metricscited by 83
  2. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 79
  3. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  4. 2025A Survey of Direct Preference Optimizationcited by 36
  5. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearningcited by 23
  6. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 22
  7. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  8. 2025LLM Unlearning with LLM Beliefscited by 17
  9. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  10. 2025A General Framework to Enhance Fine-tuning-based LLM Unlearningcited by 14
  11. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
  12. 2025GRU: Mitigating the Trade-off between Unlearning and Retention for LLMscited by 12
38 more
  1. 2025LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarkscited by 12
  2. 2025Pre-training Limited Memory Language Models with Internal and External Knowledgecited by 12
  3. 2025SALAD: Systematic Assessment of Machine Unlearning on LLM-Aided Hardware Designcited by 12
  4. 2025Distillation Robustifies Unlearningcited by 11
  5. 2025SoK: Machine Unlearning for Large Language Modelscited by 11
  6. 2025Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputscited by 11
  7. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 11
  8. 2025Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Modelscited by 10
  9. 2026Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearningcited by 10
  10. 2025Improving LLM Unlearning Robustness via Random Perturbationscited by 9
  11. 2025Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedycited by 9
  12. 2025UCD: Unlearning in LLMs via Contrastive Decodingcited by 9
  13. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 9
  14. 2025Leak@k: Unlearning Does Not Make LLMs Forget Under Probabilistic Decodingcited by 9
  15. 2025Existing Large Language Model Unlearning Evaluations Are Inconclusivecited by 8
  16. 2025Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Modelscited by 8
  17. 2025LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examplescited by 8
  18. 2026BalDRO: A Distributionally Robust Optimization based Framework for Large Language Model Unlearningcited by 8
  19. 2025Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMscited by 7
  20. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Modelscited by 7
  21. 2026Contrastive Representation Shaping for LLM Unlearningcited by 7
  22. 2026Attention Smoothing Is All You Need For Unlearningcited by 7
  23. 2026Explainable LLM Unlearning Through Reasoningcited by 7
  24. 2025Not All Tokens Are Meant to Be Forgottencited by 6
  25. 2025On the Impossibility of Retrain Equivalence in Machine Unlearningcited by 6
  26. 2025Forgetting to Forget: Attention Sink as A Gateway for Backdooring LLM Unlearningcited by 6
  27. 2026Is Gradient Ascent Really Necessary? Memorize to Forget for Machine Unlearningcited by 6
  28. 2025Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Modelscited by 5
  29. 2025Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Modelscited by 5
  30. 2025Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillationcited by 4
  31. 2025OFMU: Optimization-Driven Framework for Machine Unlearningcited by 4
  32. 2025Downgrade to Upgrade: Optimizer Simplification Enhances Robustness in LLM Unlearningcited by 4
  33. 2025Geometric-disentangelment Unlearningcited by 4
  34. 2026REBEL: Hidden Knowledge Recovery via Evolutionary-Based Evaluation Loopcited by 4
  35. 2026Gauss-Newton Unlearning for the LLM Eracited by 4
  36. 2025LetheViT: Selective Machine Unlearning for Vision Transformers via Attention-Guided Contrastive Learningcited by 3
  37. 2025Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearningcited by 3
  38. 2025Forget to Know, Remember to Use: Context-Aware Unlearning for Large Language Modelscited by 3

50 most cited of 172 All 172 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Multi-Objective Large Language Model Unlearning
  2. 2026De-attribute to Forget for LLM Unlearning
  3. 2025Rethinking LLM Unlearning Objectives: A Gradient Perspective and Go Beyond
  4. 2025LLM Unlearning using Gradient Ratio-Based Influence Estimation and Noise Injection
  5. 2024LLM Unlearning via Loss Adjustment with Only Forget Data
  6. 2024Unlearning in- vs. out-of-distribution data in LLMs under gradient-based method
  7. 2026Unlearning in LLMs: Methods, Evaluation, and Open Challenges
  8. 2025Not Every Token Needs Forgetting: Selective Unlearning to Limit Change in Utility in Large Language Model Unlearning