mlsanityData to 7 Oct 2026

Towards Understanding the Robustness of Sparse Autoencoders

arXiv 2604.18756 ↗First posted 20 Apr 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) remain vulnerable to optimization-based jailbreak attacks that exploit internal gradient structure. While Sparse Autoencoders (SAEs) are widely used for interpretability, their robustness implications remain underexplored. We present a study of integrating pretrained SAEs into transformer residual streams at inference time, without modifying model weights or blocking gradients. Across four model families (Gemma, LLaMA, Mistral, Qwen) and two strong white-box attacks (GCG, BEAST) plus three black-box benchmarks, SAE-augmented models achieve up to a 5x reduction in jailbreak success rate relative to the undefended baseline and reduce cross-model attack transferability. Parametric ablations reveal (i) a monotonic dose-response relationship between L0 sparsity and attack success rate, and (ii) a layer-dependent defense-utility tradeoff, where intermediate layers balance robustness and clean performance. These findings are consistent with a representational bottleneck hypothesis: sparse projection reshapes the optimization geometry exploited by jailbreak attacks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 25 of its 32 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

25 of 32 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing15 Jan
  2. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Models18 Apr
  3. 2025Short-length Adversarial Training Helps LLMs Defend Long-length Jailbreak Attacks: Theoretical and Empirical Evidence6 Feb
  4. 2025Smoothed Embeddings for Robust Language Models27 Jan
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey5 Jul
  6. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
  7. 2024Fast Adversarial Attacks on Language Models In One GPU Minute23 Feb
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  9. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  10. 2023Detecting Language Model Attacks with Perplexity27 Aug
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 14

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2024Scaling and evaluating sparse autoencoderscited by 530
  5. 2018Generating Natural Language Adversarial Examplescited by 422
  6. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  7. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  8. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  9. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  10. 2024MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Modelscited by 43
  11. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 42
  12. 2024Does Representation Matter? Exploring Intermediate Layers in Large Language Modelscited by 31
2 more
  1. 2024PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routingcited by 10
  2. 2025Interpreting the Latent Structure of Operator Precedence in Language Modelscited by 2

Built on by

Not cited yet.