mlsanityData to 7 Oct 2026

Guardrail Baselines for Unlearning in LLMs

arXiv 2403.03329 ↗First posted 5 Mar 2024cs.CLcited by 90

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Recent work has demonstrated that finetuning is a promising approach to 'unlearn' concepts from large language models. However, finetuning can be expensive, as it requires both generating a set of examples and running iterations of finetuning to update the model. In this work, we show that simple guardrail-based approaches such as prompting and filtering can achieve unlearning results comparable to finetuning. We recommend that researchers investigate these lightweight baselines when evaluating the performance of more computationally intensive finetuning methods. While we do not claim that methods such as prompting or filtering are universal solutions to the problem of unlearning, our work suggests the need for evaluation metrics that can better separate the power of guardrails vs. finetuning, and highlights scenarios where guardrails expose possible unintended behavior in existing metrics and benchmarks.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 44 of its 56 references that are in the map. First posted on arXiv 5 Mar 2024.

Builds on

44 of 56 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Eight Methods to Evaluate Robust Unlearning in LLMs26 Feb
  2. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  3. 2024Fast Exact Unlearning for In-Context Learning Data for LLMs1 Feb
  4. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  5. 2023Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges27 Nov
  6. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  7. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  8. 2023Fast Machine Unlearning Without Retraining Through Selective Synaptic Dampening15 Aug
  9. 2023Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutions8 Jul
  10. 2021Fast Yet Effective Machine Unlearning17 Nov
  11. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep
  12. 2021Adaptive Machine Unlearning8 Jun

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2023Mistral 7Bcited by 3,065
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  10. 2019Machine Unlearningcited by 1,020
  11. 2021Membership Inference Attacks From First Principlescited by 839
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
20 more
  1. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  2. 2019Certified Data Removal from Machine Learning Modelscited by 512
  3. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 447
  4. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  5. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  6. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  7. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  8. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  9. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  10. 2020When Machine Unlearning Jeopardizes Privacycited by 181
  11. 2023Removing RLHF Protections in GPT-4 via Fine-Tuningcited by 165
  12. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  13. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  14. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  15. 2019Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learningcited by 117
  16. 2023Calibrated Language Models Must Hallucinatecited by 114
  17. 2022Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcited by 86
  18. 2019Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answeringcited by 58
  19. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  20. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 45; 2026: 23.

90 papers.

Sort:

By area:

  1. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearningcited by 160
  2. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2024SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearningcited by 104
  5. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  6. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Modelscited by 77
  7. 2024Catastrophic Failure of LLM Unlearning via Quantizationcited by 76
  8. 2025Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyondcited by 76
  9. 2024LLM Unlearning via Loss Adjustment with Only Forget Datacited by 71
  10. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  11. 2024A Closer Look at Machine Unlearning for Large Language Modelscited by 54
  12. 2024On Large Language Model Continual Unlearningcited by 51
38 more
  1. 2024Machine Unlearning in Generative AI: A Surveycited by 49
  2. 2024Soft Prompting for Unlearning in Large Language Modelscited by 42
  3. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  4. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearningcited by 39
  5. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  6. 2025Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuningcited by 34
  7. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspectivecited by 33
  8. 2024Alternate Preference Optimization for Unlearning Factual Knowledge in Large Language Modelscited by 32
  9. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Modelscited by 30
  10. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 29
  11. 2025Representation Bending for Large Language Model Safetycited by 28
  12. 2024To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Modelscited by 25
  13. 2024WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Modelscited by 25
  14. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuningcited by 20
  15. 2025GUARD: Generation-time LLM Unlearning via Adaptive Restriction and Detectioncited by 17
  16. 2024UnStar: Unlearning with Self-Taught Anti-Sample Reasoning for LLMscited by 14
  17. 2024Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Researchcited by 14
  18. 2025A General Framework to Enhance Fine-tuning-based LLM Unlearningcited by 12
  19. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 12
  20. 2025LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarkscited by 11
  21. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  22. 2025Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completenesscited by 11
  23. 2025Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputscited by 10
  24. 2024UNLEARN Efficient Removal of Knowledge in Large Language Modelscited by 9
  25. 2025Agents Are All You Need for LLM Unlearningcited by 9
  26. 2025AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Securitycited by 9
  27. 2025UCD: Unlearning in LLMs via Contrastive Decodingcited by 9
  28. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 9
  29. 2025DRAGON: Guard LLM Unlearning in Context via Negative Detection and Reasoningcited by 7
  30. 2026Explainable LLM Unlearning Through Reasoningcited by 7
  31. 2025Improving LLM Unlearning Robustness via Random Perturbationscited by 6
  32. 2025Step-by-Step Reasoning Attack: Revealing 'Erased' Knowledge in Large Language Modelscited by 6
  33. 2026Agentic Unlearning: When LLM Agent Meets Machine Unlearningcited by 6
  34. 2026Attention Smoothing Is All You Need For Unlearningcited by 6
  35. 2025Erasing Without Remembering: Implicit Knowledge Forgetting in Large Language Modelscited by 5
  36. 2025Privacy-Enhancing Paradigms within Federated Multi-Agent Systemscited by 5
  37. 2025SEPS: A Separability Measure for Robust Unlearning in LLMscited by 5
  38. 2024LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Modelscited by 4

50 most cited of 90 All 90 (JSON)