mlsanityData to 14 Oct 2026

Mitigating Memorization in LLMs using Activation Steering

arXiv 2503.06040 ↗First posted 8 Mar 2025cs.CLcited by 8

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

The memorization of training data by Large Language Models (LLMs) poses significant risks, including privacy leaks and the regurgitation of copyrighted content. Activation steering, a technique that directly intervenes in model activations, has emerged as a promising approach for manipulating LLMs. In this work, we explore the effectiveness of activation steering in reducing memorization while preserving generalization capabilities. We conduct empirical evaluations using a controlled memorization benchmark of literary material and demonstrate that our method successfully suppresses memorized content with minimal degradation in model performance in Gemma. Additionally, we analyze the trade-offs between suppression effectiveness and linguistic fluency, highlighting the advantages and limitations of activation-based interventions. Our findings contribute to ongoing efforts in developing safer and more privacy-preserving LLMs by providing a practical and efficient mechanism to mitigate unintended memorization.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 30 of its 38 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

30 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec

Other references · 29

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2016Deep Learning with Differential Privacycited by 2,728
  3. 2020Extracting Training Data from Large Language Modelscited by 2,189
  4. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  5. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  6. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  7. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  9. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  11. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
  12. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 314
17 more
  1. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 175
  2. 2021How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVENcited by 124
  3. 2021Large-Scale Differentially Private BERTcited by 118
  4. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  5. 2024Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsalcited by 118
  6. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 92
  7. 2024How Do Large Language Models Acquire Factual Knowledge During Pretraining?cited by 90
  8. 2023Bag of Tricks for Training Data Extraction from Language Modelscited by 70
  9. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  10. 2023Exploring Memorization in Fine-tuned Language Modelscited by 45
  11. 2023Controlling the Extraction of Memorized Data from Large Language Models via Prompt-Tuningcited by 40
  12. 2024Personalized LLM Response Generation with Parameterized Memory Injectioncited by 33
  13. 2021How BPE Affects Memorization in Transformerscited by 29
  14. 2023Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarkscited by 29
  15. 2024Undesirable Memorization in Large Language Models: A Surveycited by 29
  16. 2024MemSim: A Bayesian Simulator for Evaluating Memory of LLM-based Personal Assistantscited by 24
  17. 2024"Ghost of the past": identifying and resolving privacy leakage from LLM's memory through proactive user interactioncited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 5.

8 papers.

  1. 2025The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigationcited by 10
  2. 2025ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Datacited by 6
  3. 2026Mitigating Over-Personalization in LLMs via Structured Memorycited by 3
  4. 2026Mitigating Memorization Where It Happenscited by 2
  5. 2025Randomized Masked Finetuning: An Efficient Way to Mitigate Memorization of PIIs in LLMscited by 0
  6. 2026Continuous Interpretive Steering for Scalar Diversitycited by 0
  7. 2026Output Vector Editing for Memorization Mitigation in Large Language Modelscited by 0
  8. 2026What Survives When Activation Steering Is Used as Memorycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning
  2. 2026Does Steering Break Your Model? A Multi-Dimensional Evaluation Suite for LLM Steering Methods
  3. 2023SoK: Memorization in General-Purpose Large Language Models
  4. 2024Extracting Unlearned Information from LLMs with Activation Steering
  5. 2024Mitigating Memorization In Language Models
  6. 2025Analyzing Memorization in Large Language Models through the Lens of Model Attribution
  7. 2025Hubble: a Model Suite to Advance the Study of LLM Memorization
  8. 2025Leaner Training, Lower Leakage: Revisiting Memorization in LLM Fine-Tuning with LoRA