mlsanityData to 7 Oct 2026

LLM Safety From Within: Detecting Harmful Content with Internal Representations

arXiv 2604.18519 ↗First posted 20 Apr 2026cs.AIcited by 10

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Guard models are widely used to detect harmful content in user prompts and LLM responses. However, state-of-the-art guard models rely solely on terminal-layer representations and overlook the rich safety-relevant features distributed across internal layers. We present SIREN, a lightweight guard model that harnesses these internal features. By identifying safety neurons via linear probing and combining them through an adaptive layer-weighted strategy, SIREN builds a harmfulness detector from LLM internals without modifying the underlying model. Our comprehensive evaluation shows that SIREN substantially outperforms state-of-the-art open-source guard models across multiple benchmarks while using 250 times fewer trainable parameters. Moreover, SIREN exhibits superior generalization to unseen benchmarks, naturally enables real-time streaming detection, and significantly improves inference efficiency compared to generative guard models. Overall, our results highlight LLM internal states as a promising foundation for practical, high-performance harmfulness detection.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 36 of its 45 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

36 of 45 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depth20 Oct
  2. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignment24 Jul
  3. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails15 Jan
  4. 2024Lightweight Safety Classification Using Pruned Language Models18 Dec
  5. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs26 Jun

Other references · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  7. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  8. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  9. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  10. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  11. 2017Deep Layer Aggregationcited by 436
  12. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
19 more
  1. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  2. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  3. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  4. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  5. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
  6. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 206
  7. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 190
  8. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  9. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 126
  10. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  11. 2020Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT modelcited by 87
  12. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 77
  13. 2019How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representationscited by 63
  14. 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 57
  15. 2024HSF: Defending against Jailbreak Attacks with Hidden State Filteringcited by 21
  16. 2025Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representationscited by 14
  17. 2023SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classificationcited by 10
  18. 2025Generative or Discriminative? Revisiting Text Classification in the Era of Transformerscited by 9
  19. 2025Do Internal Layers of LLMs Reveal Patterns for Jailbreak Detection?cited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

  1. 2026LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systemscited by 6
  2. 2026MINER: Mining Multimodal Internal Representation for Efficient Retrievalcited by 2
  3. 2026AERIC: Anticipatory Hidden-State Monitoring for Implicit Harmful Dialoguecited by 0
  4. 2026D2-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation Signalscited by 0
  5. 2026Tandem Reinforcement Learning with Verifiable Rewardscited by 0
  6. 2026Online Conformal Prediction Beyond Feedbackcited by 0
  7. 2026Withholding the Completing Chunk: Exact Release-Boundary Equivalence for Production Streaming Guardrailscited by 0
  8. 2026Probing the Prefill: Detecting Code Vulnerabilities via Latent Activationscited by 0
  9. 2026BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agentscited by 0
  10. 2026Agent Safety From Within: Detecting Harmful Trajectories from LLM Internal Statescited by 0