mlsanityData to 14 Oct 2026

Concepts Whisper: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations

arXiv 2605.01609 ↗First posted 2 May 2026cs.LG cs.AIcited by 0

On the map: Mechanistic interpretability → Sparse autoencoder training and evaluation

Abstract (arXiv)

We find that transformer concept representations systematically anti-concentrate in the spectral tail of the unembedding covariance, encoding word-level concepts in low-variance directions across a 17-model core suite and an expanded set of 22 semantic concept categories, with convergent replications from three independent extraction methods. Residual-stream difference-of-means vectors anti-concentrate in all 17 models (model-level one-sample t-test, p = 3.8e-9), and remain more tail-aligned than norm-matched random directions in 13 of 17; convergent support comes from sparse autoencoder (SAE) features (p = 4.5e-19 across concepts within a model) and linear probes on Llama and Qwen. We identify a dual geometry: activation-space concept directions anti-concentrate while static unembedding-row contrasts concentrate in high-variance directions (p < 10^-4). This investigation arose from testing whether the causal inner product of Park et al. (2024) aids cross-lingual concept transport; a matched-spectrum randomization across 17 models and four language pairs finds no evidence that Whitened Causal Alignment improves over spectral regularization alone (p = 0.95). Split-injection interventions, restricted to steering strengths at which both arms remain interpretable, show the predicted interference asymmetry in four of five models (paired Cohen's d_z up to 1.19) with no significant reversal inside that regime, and POS-tag probing across eight models shows syntax preferentially encoded in the high-variance subspace in six of eight architectures, with a significant reversal in the Qwen 2.5 family. These results suggest transformers rotate semantic content into spectrally quiet regions during contextualized processing, where, in some architectures, interventions may reduce grammatical disruption relative to high-variance steering.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Sparse autoencoder training and evaluation, one of 13 directions in Mechanistic interpretability. Builds on 27 of its 44 references that are in the map. First posted on arXiv 2 May 2026.

Builds on

27 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 29 Aug

Other references · 26

  1. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  6. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 784
  7. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  9. 2017Word Translation Without Parallel Datacited by 487
  10. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  11. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  12. 2019Representation Degeneration Problem in Training Natural Language Generation Modelscited by 255
14 more
  1. 2018Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learningcited by 215
  2. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  3. 2021Whitening Sentence Representations for Better Semantics and Faster Retrievalcited by 202
  4. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 181
  5. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  6. 2021All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Qualitycited by 156
  7. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
  8. 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 123
  9. 2020Predicting trends in the quality of state-of-the-art neural networks without access to training or testing datacited by 115
  10. 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 112
  11. 2019Traditional and Heavy-Tailed Self Regularization in Neural Network Modelscited by 105
  12. 2021WhiteningBERT: An Easy Unsupervised Sentence Embedding Approachcited by 69
  13. 2024Small Singular Values Matter: A Random Matrix Analysis of Transformer Modelscited by 21
  14. 2025Language-Specific Latent Process Hinders Cross-Lingual Performancecited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
  2. 2026Laguerre Geometry for Interpreting Large Language Models
  3. 2026Global Divergence, Local Convergence: Representation Geometry in SSMs and Transformers
  4. 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers
  5. 2025Uncovering the Computational Ingredients of Human-Like Representations in LLMs
  6. 2026Are You Thinking What I am Thinking? : Examining Conceptual Separation in Neural Architectures
  7. 2026A Unifying Framework for Concept-Based Representational Similarity
  8. 2025Bridging the Dimensional Chasm: Uncover Layer-wise Dimensional Reduction in Transformers through Token Correlation