mlsanityData to 14 Oct 2026

DANGERLENS: Probing Dangerous Representations Hidden by Safety Alignment

arXiv 2610.19365 ↗First posted 14 Oct 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Safety alignment teaches language models to refuse harmful requests, yet the internal accessibility of hazardous concepts can retain traces of earlier training. Can checkpoints with equally safe behavior differ in what remains accessible behind their refusals? In this work, we investigate this question through paired training histories and a direct audit of intermediate representations. We introduce DANGERLENS, a probe that maps hidden states into vocabulary space and uses blinded semantic judgments to distinguish harmful goals and operational means from hazard recognition and safety control. Experiments on three Qwen1.5 scales show that checkpoints with prior harmful exposure generally retain greater hazardous concept accessibility after the same safety alignment, with the strongest separation in intermediate and later layers. The same harmful update also produces different behavioral trajectories from closely matched starting points. Human relevance validation and readout ablations reinforce the measured accessibility contrast. Our findings show that training history remains visible inside behaviorally matched models and motivate internal accessibility audits alongside output safety evaluations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 34 of its 50 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

34 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference20 Jun
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  3. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 31

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  7. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  11. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
19 more
  1. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  3. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  4. 2023Steering Language Models With Activation Engineeringcited by 922
  5. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  6. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  7. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  8. 2019Designing and Interpreting Probes with Control Taskscited by 640
  9. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  11. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  12. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  13. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
  14. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 191
  15. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  16. 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
  17. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  18. 2024Language Models Resist Alignment: Evidence From Data Compressioncited by 53
  19. 2025InterMT: Multi-Turn Interleaved Preference Alignment with Human Feedbackcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Read the Scene, Not the Script: Outcome-Aware Safety for LLMs
  2. 2026Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection
  3. 2026Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
  4. 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
  5. 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs
  6. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons
  7. 2024Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models
  8. 2025Safety Subspaces are Not Linearly Distinct: A Fine-Tuning Case Study