mlsanityData to 7 Oct 2026

Detoxifying LLMs via Representation Erasure-Based Preference Optimization

arXiv 2602.23391 ↗First posted 24 Feb 2026cs.LGcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Large language models (LLMs) trained on webscale data can produce toxic outputs, raising concerns for safe deployment. Prior defenses, based on applications of DPO, NPO, and similar algorithms, reduce the likelihood of harmful continuations, but not robustly so: they are vulnerable to adversarial prompting and easily undone by fine-tuning-based relearning attacks. Indeed, research has shown that these edits to the model are superficial: linear probing reveals that harmful "directions" remain present in representations. To address this, we propose Representation Erasure-based Preference Optimization (REPO), reformulating detoxification as a token-level preference problem. Using a novel objective with preference data, we force the representations of toxic continuations to converge toward their benign counterparts. Our mechanistic analysis reveals that this granular approach is critical: unlike baselines, REPO induces deep, localized edits to toxicity-encoding neurons while preserving general model utility. Exhaustive evaluations show that REPO achieves state-of-the-art robustness, stopping sophisticated threats-including relearning attacks and enhanced GCG jailbreaks-where existing representation- and output-based methods fail.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 26 of its 35 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

26 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space14 Feb
  2. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity3 Jan

Other references · 24

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  3. 2016Pointer Sentinel Mixture Modelscited by 2,992
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  9. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 522
  10. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  11. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  12. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
12 more
  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  2. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  3. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  4. 2024Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearningcited by 81
  5. 2023Unveiling the Implicit Toxicity in Large Language Modelscited by 51
  6. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  7. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 21
  8. 2025Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuningcited by 20
  9. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 17
  10. 2025From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularizationcited by 12
  11. 2025Unlearning That Lasts: Utility-Preserving, Robust, and Almost Irreversible Forgetting in LLMscited by 9
  12. 2025Align-then-Unlearn: Embedding Alignment for LLM Unlearningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languages
  2. 2025Optimus: A Robust Defense Framework for Mitigating Toxicity while Fine-Tuning Conversational AI
  3. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing
  4. 2025Circumventing Safety Alignment in Large Language Models Through Embedding Space Toxicity Attenuation
  5. 2026Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
  6. 2024Large Language Models can be Strong Self-Detoxifiers
  7. 2024Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing
  8. 2024Mission Impossible: A Statistical Perspective on Jailbreaking LLMs