Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
We identify Harmfulness Propagation Dynamics (HPD): for harmful prompts, the projection of the last-token hidden state onto a learned harm direction rises monotonically with transformer depth, whereas benign prompts remain flat or oscillatory. This cross-layer signature reflects harmful intent as a progressively resolved semantic property: surface form appears early, while pragmatic intent consolidates later, making the trajectory shape more informative than any single-layer snapshot. Moreover, LDA-based harm directions, learned per layer, remain stable across random splits (pairwise cosine similarity >0.97), supporting the projection sequence as a reproducible structured signal. Building on HPD, we introduce textbfherald (Harmful Encoding Recognition via Activation Layer Dynamics). This lightweight input moderator extracts a seven-dimensional feature record, slope, curvature, monotonicity, onset layer, and related statistics from the cross-layer projection sequence and classifies it with a 288-parameter MLP. herald stores one d-dimensional direction per layer (262,KB for a 32-layer, d=4096 model), requires no gradient computation during training, and adds only 2.6×10-6 prefill FLOPs at inference. Across eight prompt-harmfulness benchmarks and four model families, herald achieves an average F1 of 89.3 on OLMo2-7B, surpassing all tested guard models on adversarial jailbreak detection (98.4 vs. 96.9 F1) and outperforming prior latent-based methods by 2.3-4.1 F1 points on every backbone. Per-instance trajectories provide machine-readable audit records that reveal when and how harmfulness emerges, offering an interpretability advantage over single-layer approaches.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 26 of its 35 references that are in the map. First posted on arXiv 11 Sep 2026.
Builds on
26 of 35 references.
Closest ancestors · 1
Same direction, newest first
Other references · 25
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2020Learning to summarize from human feedbackcited by 2,588
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2020A Primer in BERTology: What we know about how BERT workscited by 815
13 more
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
- 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
- 2023Are aligned neural networks adversarially aligned?cited by 315
- 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
- 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
- 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 216
- 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
- 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 134
- 2023SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Modelscited by 76
- 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 71
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Truth Lies Deep: Countering Semantic Camouflage via Latent Intent Verification
- 2026Hidden in Plain Sight: Advancing Harmful Content Understanding via Perception-Aware Adversarial Self-Play
- 2026The Geometry of Harmfulness in Multi-Turn Attacks
- 2025Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers
- 2025The Geometry of Harmfulness in LLMs through Subconcept Probing
- 2026Attention Is Where You Attack
- 2026Target-free Latent Safety Alignment
- 2026Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streams