mlsanityData to 7 Oct 2026

Large Language Models Encode Semantics and Alignment in Linearly Separable Representations

arXiv 2507.09709 ↗First posted 13 Jul 2025cs.CL cs.LGcited by 14

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Understanding the latent space geometry of large language models (LLMs) is key to interpreting their behavior and improving alignment. Yet it remains unclear to what extent LLMs linearly organize representations related to semantic understanding. To explore this, we conduct a large-scale empirical study of hidden representations in 11 autoregressive models across six scientific topics. We find that high-level semantic information consistently resides in low-dimensional subspaces that form linearly separable representations across domains. This separability becomes more pronounced in deeper layers and under prompts that elicit structured reasoning or alignment behaviorunicodex2013even when surface content remains unchanged. These findings motivate geometry-aware tools that operate directly in latent space to detect and mitigate harmful and adversarial content. As a proof of concept, we train an MLP probe on final-layer hidden states as a lightweight latent-space guardrail. This approach substantially improves refusal rates on malicious queries and prompt injections that bypass both the model's built-in safety alignment and external token-level filters.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 37 of its 60 references that are in the map. First posted on arXiv 13 Jul 2025.

Builds on

37 of 60 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun

Other references · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  11. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
24 more
  1. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  6. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
  7. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  8. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  9. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  10. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  11. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  12. 2023In-Context Learning Creates Task Vectorscited by 302
  13. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 293
  14. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  15. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  16. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  17. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  18. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 165
  19. 2023The geometry of hidden representations of large transformer modelscited by 108
  20. 2019On the Use of ArXiv as a Datasetcited by 101
  21. 2020Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligencecited by 81
  22. 2023The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Modelscited by 37
  23. 2025Improving Reasoning Performance in Large Language Models via Representation Engineeringcited by 21
  24. 2025Learning Task Representations from In-Context Learningcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 13.

14 papers.

Sort:

By area:

  1. 2026When Benchmarks Lie: Evaluating Malicious Prompt Classifiers Under True Distribution Shiftcited by 8
  2. 2026GeoSteer: Faithful Chain-of-Thought Steering via Latent Manifold Gradientscited by 3
  3. 2026Reasoning Fails Where Step Flow Breakscited by 2
  4. 2026Test-Time Safety Alignmentcited by 2
  5. 2026Why Self-Rewarding Works: Theoretical Guarantees for Iterative Alignment of Language Modelscited by 1
  6. 2026Hierarchical Concept Embedding & Pursuit for Interpretable Image Classificationcited by 1
  7. 2026An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammarscited by 1
  8. 2026Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Modelscited by 1
  9. 2025Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysiscited by 0
  10. 2026Multi-Task LLM with LoRA Fine-Tuning for Automated Cancer Staging and Biomarker Extractioncited by 0
  11. 2026Subliminal Clocks: Latent Time Modelling in Diffusion Language Modelscited by 0
  12. 2026Probing the Prefill: Detecting Code Vulnerabilities via Latent Activationscited by 0
2 more
  1. 2026CG-Probes: Recovering Guardrail Directions from Patient Query Embeddingscited by 0
  2. 2026MetaSteer: Context-Conditioned, nonlinear Steering via Attention-Projection Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Do Large Language Models Play Six Degrees of Separation? Measuring Topological Compression in Long-Context Manifolds
  2. 2025The Blessing and Curse of Dimensionality in Safety Alignment
  3. 2026Ventriloquist LLMs: Linear Alignment of Late-Stage Representations
  4. 2026Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States
  5. 2026Beyond Linear Concepts: Discovering and Aligning Non-Linear Concept Manifolds in Large Language Models
  6. 2026Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
  7. 2025Polarity-Aware Probing for Quantifying Latent Alignment in Language Models
  8. 2026On the Geometric Structure of Layer Updates in Deep Language Models