LMEraser: Large Model Unlearning through Adaptive Prompt Tuning
On the map: LLM safety and security → Machine unlearning
Abstract (arXiv)
To address the growing demand for privacy protection in machine learning, we propose a novel and efficient machine unlearning approach for Large Models, called LMEraser. Existing unlearning research suffers from entangled training data and complex model architectures, incurring extremely high computational costs for large models. LMEraser takes a divide-and-conquer strategy with a prompt tuning architecture to isolate data influence. The training dataset is partitioned into public and private datasets. Public data are used to train the backbone of the model. Private data are adaptively clustered based on their diversity, and each cluster is used to optimize a prompt separately. This adaptive prompt tuning mechanism reduces unlearning costs and maintains model performance. Experiments demonstrate that LMEraser achieves a 100-fold reduction in unlearning costs without compromising accuracy compared to prior work. Our code is available at: urlhttps://github.com/lmeraser/lmeraser.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 30 of its 45 references that are in the map. First posted on arXiv 17 Apr 2024.
Builds on
30 of 45 references.
Closest ancestors · 5
Same direction, newest first
- 2023Machine Unlearning: Solutions and Challenges14 Aug
- 2022Algorithms that Approximate Data Removal: New Results and Limitations25 Sep
- 2021On the Necessity of Auditable Algorithmic Definitions for Machine Unlearning22 Oct
- 2021Machine Unlearning of Features and Labels26 Aug
- 2021Remember What You Want to Forget: Algorithms for Machine Unlearning4 Mar
Other references · 25
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2022Visual Prompt Tuningcited by 1,674
13 more
- 2019Machine Unlearningcited by 1,020
- 2021Domain Generalization: A Surveycited by 760
- 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
- 2019Certified Data Removal from Machine Learning Modelscited by 512
- 2020Mixed-Privacy Forgetting in Deep Networkscited by 158
- 2022The Privacy Onion Effect: Memorization is Relativecited by 128
- 2020Machine Unlearning for Random Forestscited by 125
- 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
- 2021Federated Unlearning via Class-Discriminative Pruningcited by 108
- 2023GIF: A General Graph Unlearning Strategy via Influence Functioncited by 59
- 2023Diversity-Aware Meta Visual Promptingcited by 57
- 2022PUMA: Performance Unchanged Model Augmentation for Training Data Removalcited by 56
- 2023`A-la-carte Prompt Tuning (APT): Combining Distinct Data Via Composable Promptingcited by 17
Built on by
Citing papers per year: 2025: 2.
2 papers.