mlsanityData to 13 Oct 2026

RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning

arXiv 2512.04457 ↗First posted 4 Dec 2025cs.CLcited by 0

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Machine unlearning for large language models (LLMs) remains challenging because full retraining is costly, while approximate methods often struggle to remove targeted behaviors without degrading retained utility, especially under limited post-deployment supervision. We consider a practical PEFT setting for targeted behavioral contamination removal with a small forget set, a limited retain buffer, and LoRA-only updates, and propose RapidUn, an influence-guided framework that converts cross-sample influence estimates into fixed sample-specific weights for weighted LoRA unlearning. Across Llama-3-8B on Dolly-15k and Alpaca-57k, with cross-model validation on Mistral-7B + Dolly-15k, RapidUn achieves lower seen-trigger and OOD-trigger-family ASR than Fisher, GA, and LoReUn while maintaining competitive clean utility. On Llama-3-8B + Alpaca-57k, it achieves a 77x wall-clock speedup over the clean-corpus LoRA retraining reference. Complementary TOFU, semantic LLM-judge, and IFEval evaluations further support the effectiveness of influence-guided sample reweighting beyond the controlled trigger benchmark.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 50 of its 70 references that are in the map. First posted on arXiv 4 Dec 2025.

Builds on

50 of 70 references.

Closest ancestors · 22

Same direction, newest first

  1. 2025SoK: Machine Unlearning for Large Language Models10 Jun
  2. 2025BLUR: A Bi-Level Optimization Approach for LLM Unlearning9 Jun
  3. 2025Existing Large Language Model Unlearning Evaluations Are Inconclusive31 May
  4. 2025Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy31 May
  5. 2025Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning17 May
  6. 2025OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models7 May
  7. 2025LUME: LLM Unlearning with Multitask Evaluations20 Feb
  8. 2025ReLearn: Unlearning via Learning for Large Language Models16 Feb
  9. 2024A Closer Look at Machine Unlearning for Large Language Models10 Oct
  10. 2024Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning9 Oct
  11. 2024On Effects of Steering Latent Representation for Large Language Model Unlearning12 Aug
  12. 2024Revisiting Who's Harry Potter: Towards Targeted Unlearning from a Causal Intervention Perspective24 Jul
10 more
  1. 2024Soft Prompting for Unlearning in Large Language Models17 Jun
  2. 2024RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models16 Jun
  3. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methods2 Apr
  4. 2024Machine Unlearning of Pre-trained Large Language Models23 Feb
  5. 2024Towards Safer Large Language Models through Machine Unlearning15 Feb
  6. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  7. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  8. 2023DeepClean: Machine Unlearning on the Cheap by Resetting Privacy Sensitive Weights using the Fisher Diagonal17 Nov
  9. 2022A Survey of Machine Unlearning6 Sep
  10. 2021Unrolling SGD: Understanding Factors Influencing Machine Unlearning27 Sep

Other references · 28

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  6. 2013Intriguing properties of neural networkscited by 5,137
  7. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  8. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,213
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,898
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,650
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,273
  12. 2020Extracting Training Data from Large Language Modelscited by 2,181
16 more
  1. 2017Understanding Black-box Predictions via Influence Functionscited by 1,729
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,666
  3. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,240
  4. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,191
  5. 2019Machine Unlearningcited by 1,056
  6. 2022Mass-Editing Memory in a Transformercited by 990
  7. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 891
  8. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 572
  9. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 517
  10. 2019Making AI Forget You: Data Deletion in Machine Learningcited by 462
  11. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
  12. 2020Influence Functions in Deep Learning Are Fragilecited by 249
  13. 2025Extracting memorized pieces of (copyrighted) books from open-weight language modelscited by 29
  14. 2024Token-wise Influential Training Data Retrieval for Large Language Modelscited by 19
  15. 2025Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Checkcited by 5
  16. 2025LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM Unlearning using Gradient Ratio-Based Influence Estimation and Noise Injection
  2. 2023In-Context Unlearning: Language Models as Few Shot Unlearners
  3. 2026Unlearning in LLMs: Methods, Evaluation, and Open Challenges
  4. 2025Lacuna Inc. at SemEval-2025 Task 4: LoRA-Enhanced Influence-Based Unlearning for LLMs
  5. 2025LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
  6. 2023Large Language Model Unlearning
  7. 2026Inference-Time Machine Unlearning via Gated Activation Redirection
  8. 2026UnlearningSoup: Is Repeated Tuning Necessary for Large Language Model Unlearning?