# A Language Model's Guide Through Latent Space

arXiv: https://arxiv.org/abs/2402.14433 · HTML: https://mlsanity.com/paper/2402.14433/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Studying Large Language Model Generalization with Influence Functions](https://mlsanity.com/paper/2308.03296/) (2023)
- [Preference Ranking Optimization for Human Alignment](https://mlsanity.com/paper/2306.17492/) (2023)
- [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023)
- [Adversarial Attacks and Defences: A Survey](https://mlsanity.com/paper/1810.00069/) (2018)
- [Predictability and Surprise in Large Generative Models](https://mlsanity.com/paper/2202.07785/) (2022)
- [Chess as a Testbed for Language Model State Tracking](https://mlsanity.com/paper/2102.13249/) (2021)
- [Eliciting Latent Knowledge from Quirky Language Models](https://mlsanity.com/paper/2312.01037/) (2023)
- [Specific versus General Principles for Constitutional AI](https://mlsanity.com/paper/2310.13798/) (2023)
- [The Consensus Game: Language Model Generation via Equilibrium Search](https://mlsanity.com/paper/2310.09139/) (2023)
- [Do Neural Language Models Show Preferences for Syntactic Formalisms?](https://mlsanity.com/paper/2004.14096/) (2020)

## Built on by (most cited first, 30 of 43)

- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [LLMs Encode Harmfulness and Refusal Separately](https://mlsanity.com/paper/2507.11878/) (2025)
- [Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment](https://mlsanity.com/paper/2501.19309/) (2025)
- [Angular Steering: Behavior Control via Rotation in Activation Space](https://mlsanity.com/paper/2510.26243/) (2025)
- [FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering](https://mlsanity.com/paper/2504.14492/) (2025)
- [Linear Representations of Political Perspective Emerge in Large Language Models](https://mlsanity.com/paper/2503.02080/) (2025)
- [Representation Bending for Large Language Model Safety](https://mlsanity.com/paper/2504.01550/) (2025)
- [To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models](https://mlsanity.com/paper/2510.13290/) (2025)
- [Householder Pseudo-Rotation: A Novel Approach to Activation Editing in LLMs with Direction-Magnitude Perspective](https://mlsanity.com/paper/2409.10053/) (2024)
- [Controllable Context Sensitivity and the Knob Behind It](https://mlsanity.com/paper/2411.07404/) (2024)
- [Extracting Unlearned Information from LLMs with Activation Steering](https://mlsanity.com/paper/2411.02631/) (2024)
- [Representation Engineering for Large-Language Models: Survey and Research Challenges](https://mlsanity.com/paper/2502.17601/) (2025)
- [Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering](https://mlsanity.com/paper/2505.12025/) (2025)
- [Towards Inference-time Category-wise Safety Steering for Large Language Models](https://mlsanity.com/paper/2410.01174/) (2024)
- [Beyond Token Probes: Hallucination Detection via Activation Tensors with ACT-ViT](https://mlsanity.com/paper/2510.00296/) (2025)
- [Differentially Private Steering for Large Language Model Alignment](https://mlsanity.com/paper/2501.18532/) (2025)
- [LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing](https://mlsanity.com/paper/2505.16491/) (2025)
- [Activation Steering with a Feedback Controller](https://mlsanity.com/paper/2510.04309/) (2025)
- [Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought](https://mlsanity.com/paper/2510.24941/) (2025)
- [FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute](https://mlsanity.com/paper/2502.20126/) (2025)
- [SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMs](https://mlsanity.com/paper/2506.04250/) (2025)
- [Designing Role Vectors to Improve LLM Inference Behaviour](https://mlsanity.com/paper/2502.12055/) (2025)
- [From Emergence to Control: Probing and Modulating Self-Reflection in Language Models](https://mlsanity.com/paper/2506.12217/) (2025)
- [Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions](https://mlsanity.com/paper/2509.23782/) (2025)
- [Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots](https://mlsanity.com/paper/2504.03735/) (2025)
- [ThoughtProbe: Classifier-Guided Thought Space Exploration Leveraging LLM Intrinsic Reasoning](https://mlsanity.com/paper/2504.06650/) (2025)
- [Probing the Robustness of Large Language Models Safety to Latent Perturbations](https://mlsanity.com/paper/2506.16078/) (2025)
- [AdaptiveK: Complexity-Driven Sparse Autoencoders for Interpretable Language Model Representations](https://mlsanity.com/paper/2508.17320/) (2025)
- [A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering](https://mlsanity.com/paper/2510.01246/) (2025)
- [Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs](https://mlsanity.com/paper/2602.01600/) (2026)

## Abstract (arXiv)

Concept guidance has emerged as a cheap and simple way to control the behavior of language models by probing their hidden representations for concept vectors and using them to perturb activations at inference time. While the focus of previous work has largely been on truthfulness, in this paper we extend this framework to a richer set of concepts such as appropriateness, humor, creativity and quality, and explore to what degree current detection and guidance strategies work in these challenging settings. To facilitate evaluation, we develop a novel metric for concept guidance that takes into account both the success of concept elicitation as well as the potential degradation in fluency of the guided model. Our extensive experiments reveal that while some concepts such as truthfulness more easily allow for guidance with current techniques, novel concepts such as appropriateness or humor either remain difficult to elicit, need extensive tuning to work, or even experience confusion. Moreover, we find that probes with optimal detection accuracies do not necessarily make for the optimal guides, contradicting previous observations for truthfulness. Our work warrants a deeper investigation into the interplay between detectability, guidability, and the nature of the concept, and we hope that our rich experimental test-bed for guidance research inspires stronger follow-up approaches.