mlsanityData to 14 Oct 2026

Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs

arXiv 2406.11780 ↗First posted 17 Jun 2024cs.LG cs.AI cs.CLcited by 18

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Large language models (LLMs) have shown to pose social and ethical risks such as generating toxic language or facilitating malicious use of hazardous knowledge. Machine unlearning is a promising approach to improve LLM safety by directly removing harmful behaviors and knowledge. In this paper, we propose "SPlit, UNlearn, MerGE" (SPUNGE), a framework that can be used with any unlearning method to amplify its effectiveness. SPUNGE leverages data attributes during unlearning by splitting unlearning data into subsets based on specific attribute values, unlearning each subset separately, and merging the unlearned models. We empirically demonstrate that SPUNGE significantly improves the performance of two recent unlearning methods on state-of-the-art LLMs while maintaining their general capabilities on standard academic benchmarks.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 38 of its 52 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

38 of 52 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  2. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  3. 2023KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment11 May
  4. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct
  5. 2022Quark: Controllable Text Generation with Reinforced Unlearning26 May
  6. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep

Other references · 32

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  10. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  11. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  12. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
20 more
  1. 2022Editing Models with Task Arithmeticcited by 1,120
  2. 2019Machine Unlearningcited by 1,061
  3. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  4. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  5. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  6. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  7. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 618
  8. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  9. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  10. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  11. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 465
  12. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
  13. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  14. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 404
  15. 2020Amnesiac Machine Learningcited by 319
  16. 2023Are aligned neural networks adversarially aligned?cited by 315
  17. 2020Approximate Data Deletion from Machine Learning Modelscited by 255
  18. 2023Pretraining Language Models with Human Preferencescited by 247
  19. 2020Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observationscited by 180
  20. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 156

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 10; 2026: 4.

18 papers.

Sort:

By area:

  1. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearningcited by 172
  2. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 123
  3. 2025Exact Unlearning of Finetuning Data via Model Merging at Scalecited by 14
  4. 2025Continual Unlearning for Text-to-Image Diffusion Models: A Regularization Perspectivecited by 10
  5. 2025UCD: Unlearning in LLMs via Contrastive Decodingcited by 9
  6. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 6
  7. 2025Mitigating Biases in Language Models via Bias Unlearningcited by 6
  8. 2025A Survey on Unlearning in Large Language Modelscited by 4
  9. 2026Visual-Guided Key-Token Regularization for Multimodal Large Language Model Unlearningcited by 3
  10. 2025ZJUKLAB at SemEval-2025 Task 4: Unlearning via Model Mergingcited by 2
  11. 2025Ready2Unlearn: A Learning-Time Approach for Preparing Models with Future Unlearning Readinesscited by 2
  12. 2025Revisiting the Past: Data Unlearning with Model State Historycited by 2
6 more
  1. 2026Per-parameter Task Arithmetic for Unlearning in Large Language Modelscited by 2
  2. 2025Layered Unlearning for Adversarial Relearningcited by 1
  3. 2024Deep Model Merging: The Sister of Neural Network Interpretability -- A Surveycited by 0
  4. 2025Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniquescited by 0
  5. 2026EvoMU: Evolutionary Machine Unlearningcited by 0
  6. 2026Model Unlearning Objectives Vary for Distinct Language Functionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
  2. 2026Unlearning in LLMs: Methods, Evaluation, and Open Challenges
  3. 2024Towards Effective Evaluations and Comparisons for LLM Unlearning Methods
  4. 2024The Frontier of Data Erasure: Machine Unlearning for Large Language Models
  5. 2025OpenUnlearning: Accelerating LLM Unlearning via Unified Benchmarking of Methods and Metrics
  6. 2024Rethinking Machine Unlearning for Large Language Models
  7. 2024Machine Unlearning in Large Language Models
  8. 2024Unified Parameter-Efficient Unlearning for LLMs