mlsanityData to 7 Oct 2026

REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space

arXiv 2406.09325 ↗First posted 13 Jun 2024cs.CLcited by 12

On the map: LLM safety and security → Machine unlearning

Abstract (arXiv)

Language models (LMs) risk inadvertently memorizing and divulging sensitive or personally identifiable information (PII) seen in training data, causing privacy concerns. Current approaches to address this issue involve costly dataset scrubbing, or model filtering through unlearning and model editing, which can be bypassed through extraction attacks. We propose REVS, a novel non-gradient-based method for unlearning sensitive information from LMs. REVS identifies and modifies a small subset of neurons relevant for constituent tokens that form sensitive information. To adequately evaluate our method on truly sensitive information, we curate three datasets: email and URL datasets naturally memorized by the models, and a synthetic social security number dataset that we tune the models to memorize. Compared to other methods, REVS demonstrates superior performance in unlearning sensitive information and robustness to extraction attacks, while retaining underlying model integrity.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 48 of its 79 references that are in the map. First posted on arXiv 13 Jun 2024.

Builds on

48 of 79 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning8 Apr
  2. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  3. 2024TOFU: A Task of Fictitious Unlearning for LLMs11 Jan
  4. 2023Large Language Model Unlearning14 Oct
  5. 2023In-Context Unlearning: Language Models as Few Shot Unlearners11 Oct
  6. 2023Who's Harry Potter? Approximate Unlearning in LLMs3 Oct
  7. 2023Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-428 Apr
  8. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Models4 Oct

Other references · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2016Deep Learning with Differential Privacycited by 2,682
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  8. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2019Language Models as Knowledge Bases?cited by 1,922
  11. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  12. 2019Machine Unlearningcited by 1,020
28 more
  1. 2022Mass-Editing Memory in a Transformercited by 976
  2. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  3. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  4. 2023Extracting Training Data from Diffusion Modelscited by 723
  5. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  6. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  7. 2021Editing Factual Knowledge in Language Modelscited by 548
  8. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  9. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  10. 2021Large Language Models Can Be Strong Differentially Private Learnerscited by 405
  11. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  12. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  13. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  14. 2020Descent-to-Delete: Gradient-Based Methods for Machine Unlearningcited by 303
  15. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  16. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  17. 2020Modifying Memories in Transformer Modelscited by 222
  18. 2022What Does it Mean for a Language Model to Preserve Privacy?cited by 194
  19. 2023Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attackscited by 156
  20. 2021Counterfactual Memorization in Neural Language Modelscited by 155
  21. 2022Analyzing Transformers in Embedding Spacecited by 131
  22. 2021Large-Scale Differentially Private BERTcited by 117
  23. 2022Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacycited by 97
  24. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  25. 2023DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Modelscited by 75
  26. 2020Training Production Language Models without Memorizing User Datacited by 68
  27. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreakscited by 45
  28. 2023Privacy Side Channels in Machine Learning Systemscited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 1.

12 papers.

  1. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 113
  2. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  3. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Stepscited by 43
  4. 2025Precise In-Parameter Concept Erasure in Large Language Modelscited by 11
  5. 2025CRISP: Persistent Concept Unlearning via Sparse Autoencoderscited by 9
  6. 2025Does Machine Unlearning Truly Remove Knowledge?cited by 7
  7. 2025Position: Bridge the Gaps between Machine Unlearning and AI Regulationcited by 5
  8. 2025Soft Token Attacks Cannot Reliably Audit Unlearning in Large Language Modelscited by 2
  9. 2025A Survey on Unlearning in Large Language Modelscited by 2
  10. 2025AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Modelscited by 2
  11. 2026The Unlearning Mirage: A Dynamic Framework for Evaluating LLM Unlearningcited by 1
  12. 2025Cyber for AI at SemEval-2025 Task 4: Forgotten but Not Lost: The Balancing Act of Selective Unlearning in Large Language Modelscited by 0