mlsanityData to 7 Oct 2026

Obfuscated Activations Bypass LLM Latent-Space Defenses

arXiv 2412.09565 ↗First posted 12 Dec 2024cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

Recent latent-space monitoring techniques have shown promise as defenses against LLM attacks. These defenses act as scanners that seek to detect harmful activations before they lead to undesirable actions. This prompts the question: Can models execute harmful behavior via inconspicuous latent states? Here, we study such obfuscated activations. We show that state-of-the-art latent-space defenses -- including sparse autoencoders, representation probing, and latent OOD detection -- are all vulnerable to obfuscated activations. For example, against probes trained to classify harmfulness, our attacks can often reduce recall from 100% to 0% while retaining a 90% jailbreaking rate. However, obfuscation has limits: we find that on a complex task (writing SQL code), obfuscation reduces model performance. Together, our results demonstrate that neural activations are highly malleable: we can reshape activation patterns in a variety of ways, often while preserving a network's behavior. This poses a fundamental challenge to latent-space defenses.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Dec 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 23.

40 papers.

Sort:

By area:

  1. 2025Detecting High-Stakes Interactions with Activation Probescited by 45
  2. 2025Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activationscited by 37
  3. 2025Probing and Steering Evaluation Awareness of Language Modelscited by 31
  4. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 21
  5. 2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 19
  6. 2025Benchmarking Deception Probes via Black-to-White Performance Boostscited by 17
  7. 2025STACK: Adversarial Attacks on LLM Safeguard Pipelinescited by 14
  8. 2025Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectivescited by 13
  9. 2025Difficulties with Evaluating a Deception Detector for AIscited by 11
  10. 2025Benchmarking Misuse Mitigation Against Covert Adversariescited by 8
  11. 2025Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Unseen Activation Monitorscited by 7
  12. 2025Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitorscited by 6
28 more
  1. 2026The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probescited by 6
  2. 2026Calibrate-Then-Delegate: Safety Monitoring with Risk and Budget Guarantees via Model Cascadescited by 3
  3. 2026Segment-Level Coherence for Robust Harmful Intent Probing in LLMscited by 3
  4. 2025StructTransform: A Scalable Attack Surface for Safety-Aligned Large Language Modelscited by 2
  5. 2025Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformerscited by 2
  6. 2025Probe-based Fine-tuning for Reducing Toxicitycited by 2
  7. 2026Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representationscited by 2
  8. 2026When Behavioral Safety Evaluation Fails: A Representation-Level Perspectivecited by 2
  9. 2026Train the Model, Not the Reader: Decodability Supervision for Verifiable Activation Explanationscited by 2
  10. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  11. 2026TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tamperingcited by 1
  12. 2026Trojan-Speak: Bypassing Constitutional Classifiers with No Jailbreak Tax via Adversarial Finetuningcited by 1
  13. 2026Mechanistic Anomaly Detection via Functional Attributioncited by 1
  14. 2026Reference-Grafting Matches Fine-Tuning at Eliciting Sandbagged Capabilitiescited by 1
  15. 2025Red-teaming Activation Probes using Prompted LLMscited by 0
  16. 2026Stress-Testing Alignment Audits With Prompt-Level Strategic Deceptioncited by 0
  17. 2026Adaptively Robust LLM Monitoring via Activation Watermarkingcited by 0
  18. 2026Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMscited by 0
  19. 2026PRISM: Recovering Instruction Sets from Language Model Activationscited by 0
  20. 2026Fuzzing Large Language Models to Elicit Hidden Behaviourscited by 0
  21. 2026A False Average: Pooled CoT-Monitor Accuracy Conceals a Reasoning-Dependent Fragilitycited by 0
  22. 2026Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?cited by 0
  23. 2026Measuring Activation Control in Large Language Modelscited by 0
  24. 2026A Lie Detector Test for Language Models: Reading Knowledge a Model Won't Revealcited by 0
  25. 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0
  26. 2026EADC: Evaluation of Advanced and Deep-level Compliance in Large Language Modelscited by 0
  27. 2026When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safetycited by 0
  28. 2026LLMs Learn to Evade Latent Monitors from Prior Feedback Alonecited by 0