mlsanityData to 7 Oct 2026

Representation Bending for Large Language Model Safety

arXiv 2504.01550 ↗First posted 2 Apr 2025cs.LG cs.CL cs.CRcited by 28

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Large Language Models (LLMs) have emerged as powerful tools, but their inherent safety risks - ranging from harmful content generation to broader societal harms - pose significant challenges. These risks can be amplified by the recent adversarial attacks, fine-tuning vulnerabilities, and the increasing deployment of LLMs in high-stakes environments. Existing safety-enhancing techniques, such as fine-tuning with human feedback or adversarial training, are still vulnerable as they address specific threats and often fail to generalize across unseen attacks, or require manual system-level defenses. This paper introduces RepBend, a novel approach that fundamentally disrupts the representations underlying harmful behaviors in LLMs, offering a scalable solution to enhance (potentially inherent) safety. RepBend brings the idea of activation steering - simple vector arithmetic for steering model's behavior during inference - to loss-based fine-tuning. Through extensive evaluation, RepBend achieves state-of-the-art performance, outperforming prior methods such as Circuit Breaker, RMU, and NPO, with up to 95% reduction in attack success rates across diverse jailbreak benchmarks, all with negligible reduction in model usability and general capabilities.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 65 of its 90 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

65 of 90 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Does Refusal Training in LLMs Generalize to the Past Tense?16 Jul
  2. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs26 Jun
  3. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  4. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  5. 2024Improving Alignment and Robustness with Circuit Breakers6 Jun
  6. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuning23 May
  7. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vector18 Apr
  8. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning5 Mar
  9. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications7 Feb
  10. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 54

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
38 more
  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  5. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  6. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
  7. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  8. 2023Steering Language Models With Activation Engineeringcited by 824
  9. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  11. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 507
  12. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 484
  13. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  14. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  15. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 374
  16. 2023Who's Harry Potter? Approximate Unlearning in LLMscited by 366
  17. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  18. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  19. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  20. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  21. 2023An Overview of Catastrophic AI Riskscited by 214
  22. 2022Predictability and Surprise in Large Generative Modelscited by 209
  23. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  24. 2024ReFT: Representation Finetuning for Language Modelscited by 186
  25. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  26. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  27. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  28. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 121
  29. 2024An Adversarial Perspective on Machine Unlearning for AI Safetycited by 113
  30. 2024Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimizationcited by 112
  31. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 99
  32. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  33. 2024Guardrail Baselines for Unlearning in LLMscited by 90
  34. 2024Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Spacecited by 88
  35. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 64
  36. 2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 60
  37. 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
  38. 2023Bypassing the Safety Training of Open-Source LLMs with Priming Attackscited by 48

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 19.

28 papers.

Sort:

By area:

  1. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
  2. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  3. 2025Consistency Training Helps Stop Sycophancy and Jailbreakscited by 13
  4. 2025A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Modelscited by 9
  5. 2025Improving Large Language Model Safety with Contrastive Representation Learningcited by 7
  6. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
  7. 2025Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injectioncited by 5
  8. 2026TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modelingcited by 5
  9. 2025DUSK: Do Not Unlearn Shared Knowledgecited by 4
  10. 2026Adversarial Contrastive Learning for LLM Quantization Attackscited by 4
  11. 2026GAVEL: Towards Rule-Based Safety Through Activation Monitoringcited by 4
  12. 2026Minimizing Collateral Damage in Activation Steeringcited by 4
16 more
  1. 2026Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Typescited by 3
  2. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  3. 2026TrojanPraise: Jailbreak LLMs via Benign Fine-Tuningcited by 1
  4. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Modelscited by 1
  5. 2026Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuningcited by 1
  6. 2026RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMscited by 1
  7. 2026DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuningcited by 1
  8. 2025Understanding and Mitigating Over-refusal for Large Language Models via Representation Interventioncited by 0
  9. 2026Analysing LLM Persona Generation and Fairness Interpretation in Polarised Geopolitical Contextscited by 0
  10. 2026Adaptive Probe-based Steering for Robust LLM Jailbreakingcited by 0
  11. 2026LoMC: Localized Multidirectional Correction for Refusal Suppression in Routed Foundation Modelscited by 0
  12. 2026HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignmentcited by 0
  13. 2026Mitigating Scaffolding Collapse in Socratic Tutors via Representation Alignmentcited by 0
  14. 2026Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliterationcited by 0
  15. 2026SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMscited by 0
  16. 2026How Narrative Wrapping Affects LLM Refusal: A Cross-Language Benchmark and Defensecited by 0