mlsanityData to 7 Oct 2026

Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers

arXiv 2507.09406 ↗First posted 12 Jul 2025cs.LG cs.AIcited by 2

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Large language models (LLMs) aligned for safety through techniques like reinforcement learning from human feedback (RLHF) often exhibit emergent deceptive behaviors, where outputs appear compliant but subtly mislead or omit critical information. This paper introduces adversarial activation patching, a novel mechanistic interpretability framework that leverages activation patching as an adversarial tool to induce, detect, and mitigate such deception in transformer-based models. By sourcing activations from "deceptive" prompts and patching them into safe forward passes at specific layers, we simulate vulnerabilities and quantify deception rates. Through toy neural network simulations across multiple scenarios (e.g., 1000 trials per setup), we demonstrate that adversarial patching increases deceptive outputs to 23.9% from a 0% baseline, with layer-specific variations supporting our hypotheses. We propose six hypotheses, including transferability across models, exacerbation in multimodal settings, and scaling effects. An expanded literature review synthesizes over 20 key works in interpretability, deception, and adversarial attacks. Mitigation strategies, such as activation anomaly detection and robust fine-tuning, are detailed, alongside ethical considerations and future research directions. This work advances AI safety by highlighting patching's dual-use potential and provides a roadmap for empirical studies on large-scale models.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 15 of its 26 references that are in the map. First posted on arXiv 12 Jul 2025.

Builds on

15 of 26 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  6. 2023Language Models Represent Space and Timecited by 306
  7. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  8. 2023AI Deception: A Survey of Examples, Risks, and Potential Solutionscited by 185
  9. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  10. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  11. 2024Truth is Universal: Robust Detection of Lies in LLMscited by 78
  12. 2024Obfuscated Activations Bypass LLM Latent-Space Defensescited by 40
3 more
  1. 2024A Survey on Large Language Model Hallucination via a Creativity Perspectivecited by 38
  2. 2024JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Modelscited by 7
  3. 2024Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Languagecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Probing the Robustness of Large Language Models Safety to Latent Perturbations
  2. 2026The Persistent Vulnerability of Aligned AI Systems
  3. 2025D-REX: A Benchmark for Detecting Deceptive Reasoning in Large Language Models
  4. 2025Caught in the Act: a mechanistic approach to detecting deception
  5. 2026Amnesia: Adversarial Semantic Layer Specific Activation Steering in Large Language Models
  6. 2026Detect and Suppress: A Mechanistic Defense against Adversarial Patches in VLA Models
  7. 2025Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models
  8. 2026ARREST: Adversarial Resilient Regulation Enhancing Safety and Truth in Large Language Models