# Beyond Single Concept Vector: Modeling Concept Subspace in LLMs with Gaussian Distribution

arXiv: https://arxiv.org/abs/2410.00153 · HTML: https://mlsanity.com/paper/2410.00153/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 40 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Poincar'e Embeddings for Learning Hierarchical Representations](https://mlsanity.com/paper/1705.08039/) (2017)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering](https://mlsanity.com/paper/2311.06668/) (2023)
- [A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity](https://mlsanity.com/paper/2401.01967/) (2024)
- [Linear Representations of Sentiment in Large Language Models](https://mlsanity.com/paper/2310.15154/) (2023)
- [The Geometry of Categorical and Hierarchical Concepts in Large Language Models](https://mlsanity.com/paper/2406.01506/) (2024)
- [Detoxifying Large Language Models via Knowledge Editing](https://mlsanity.com/paper/2403.14472/) (2024)
- [A Primer on the Inner Workings of Transformer-based Language Models](https://mlsanity.com/paper/2405.00208/) (2024)
- [Adaptive Activation Steering: A Tuning-Free LLM Truthfulness Improvement Method for Diverse Hallucinations Categories](https://mlsanity.com/paper/2406.00034/) (2024)
- [Improving Activation Steering in Language Models with Mean-Centring](https://mlsanity.com/paper/2312.03813/) (2023)
- [Style Vectors for Steering Generative Large Language Model](https://mlsanity.com/paper/2402.01618/) (2024)
- [Probing BERT in Hyperbolic Spaces](https://mlsanity.com/paper/2104.03869/) (2021)
- [Language Models as Hierarchy Encoders](https://mlsanity.com/paper/2401.11374/) (2024)

## Built on by (most cited first, 27 of 27)

- [A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models](https://mlsanity.com/paper/2503.05613/) (2025)
- [SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models](https://mlsanity.com/paper/2502.11356/) (2025)
- [SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models](https://mlsanity.com/paper/2505.16188/) (2025)
- [Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainability](https://mlsanity.com/paper/2501.01346/) (2025)
- [Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steering](https://mlsanity.com/paper/2505.15038/) (2025)
- [Cell-Based Representation of Relational Binding in Language Models](https://mlsanity.com/paper/2604.19052/) (2026)
- [Depth-Wise Activation Steering for Honest Language Models](https://mlsanity.com/paper/2512.07667/) (2025)
- [Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight](https://mlsanity.com/paper/2509.24169/) (2025)
- [The Lattice Representation Hypothesis of Large Language Models](https://mlsanity.com/paper/2603.01227/) (2026)
- [When is Your LLM Steerable?](https://mlsanity.com/paper/2606.11599/) (2026)
- [A Low-Rank Subspace Analysis of LLM Interventions](https://mlsanity.com/paper/2606.14388/) (2026)
- [Neuron-Level Differentiation of Memorization and Generalization in Large Language Models](https://mlsanity.com/paper/2412.18497/) (2024)
- [On Optimal Steering to Achieve Exact Fairness](https://mlsanity.com/paper/2509.15759/) (2025)
- [Localizing Task Recognition and Task Learning in In-Context Learning via Attention Head Analysis](https://mlsanity.com/paper/2509.24164/) (2025)
- [Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization](https://mlsanity.com/paper/2511.19131/) (2025)
- [Continuous sentiment scores for literary and multilingual contexts](https://mlsanity.com/paper/2508.14620/) (2025)
- [RAPTOR: Ridge-Adaptive Logistic Probes](https://mlsanity.com/paper/2602.00158/) (2026)
- [ShallowJail: Steering Jailbreaks against Large Language Models](https://mlsanity.com/paper/2602.07107/) (2026)
- [Fine-Grained Interpretation of Political Opinions in Large Language Models](https://mlsanity.com/paper/2506.04774/) (2025)
- [Interpreting Multi-Attribute Confounding through Numerical Attributes in Large Language Models](https://mlsanity.com/paper/2511.04053/) (2025)
- [Is Sentiment Banana-Shaped? Exploring the Geometry and Portability of Sentiment Concept Vectors](https://mlsanity.com/paper/2601.07995/) (2026)
- [Functional Subspace, where language models can use vector algebra to solve problems](https://mlsanity.com/paper/2602.01687/) (2026)
- [Causal Probing for Internal Visual Representations in Multimodal Large Language Models](https://mlsanity.com/paper/2605.05593/) (2026)
- [Laguerre Geometry for Interpreting Large Language Models](https://mlsanity.com/paper/2607.10578/) (2026)
- [High-probability guarantees for linear accessibility in feature superposition](https://mlsanity.com/paper/2609.09556/) (2026)
- [Usage-Modulated Sentiment Representations in Large Language Models](https://mlsanity.com/paper/2610.05069/) (2026)
- [Learning to Decide, Not to Reason: Parameter-Efficient Decision Operators via Low-Rank Activation Steering](https://mlsanity.com/paper/2610.06950/) (2026)

## Abstract (arXiv)

Probing learned concepts in large language models (LLMs) is crucial for understanding how semantic knowledge is encoded internally. Training linear classifiers on probing tasks is a principle approach to denote the vector of a certain concept in the representation space. However, the single vector identified for a concept varies with both data and training, making it less robust and weakening its effectiveness in real-world applications. To address this challenge, we propose an approach to approximate the subspace representing a specific concept. Built on linear probing classifiers, we extend the concept vectors into Gaussian Concept Subspace (GCS). We demonstrate GCS's effectiveness through measuring its faithfulness and plausibility across multiple LLMs with different sizes and architectures. Additionally, we use representation intervention tasks to showcase its efficacy in real-world applications such as emotion steering. Experimental results indicate that GCS concept vectors have the potential to balance steering performance and maintaining the fluency in natural language generation tasks.