mlsanityData to 7 Oct 2026

Before the Last Token: Diagnosing Final-Token Safety Probe Failures

arXiv 2605.12726 ↗First posted 12 May 2026cs.LGcited by 3

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Final-token safety probes monitor a single hidden state after prompt prefill, but jailbreak prompts can contain probe-visible unsafe evidence distributed across earlier user-token representations that is missed by this readout. We study this prefill-time failure mode using SafeSwitch-style probes trained only on clean harmful and benign prompts across three instruction-tuned LLMs. The probes achieve high recall on clean harmful prompts, but miss many jailbreaks and can produce false positives on safety-adjacent benign prompts. Subspace analyses suggest that missed jailbreaks differ from clean benign prompts along directions that are poorly captured by the probe's representational subspace, and increasing probe bottleneck width does not reliably resolve this mismatch. Token-level prefill analyses reveal that probe-visible unsafe evidence often appears earlier in the sequence but is not exposed at the final-token readout, while naive max-pooling over token positions overfires on safe prompts. A simple PCA-HMM trajectory model, trained only on the same clean split, recovers many final-token misses from user-content prefill trajectories without the catastrophic false-positive behavior of naive token pooling, motivating trajectory-aware hidden-state analyses as diagnostic complements to final-token probes

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 12 of its 14 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

12 of 14 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026TrajGuard: Streaming Hidden-state Trajectory Detection for Decoding-time Jailbreak Defense9 Apr
  2. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 9

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  3. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  4. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  5. 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directionscited by 38
  6. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signalscited by 29
  7. 2026Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoningcited by 9
  8. 2025The Geometry of Harmfulness in LLMs through Subconcept Probingcited by 7
  9. 2025N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluatorcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Familiescited by 2
  2. 2026Breaking Refusal in the First Half: A Mechanistic Study of the Prefill Jailbreakcited by 0
  3. 2026Hard Negatives Reveal What Easy Negatives Hide: Cross-Lingual Harmfulness Representations Degrade with Resource Tier Under Hard Negativescited by 0