mlsanityData to 7 Oct 2026

The Information Geometry of Softmax: Probing and Steering

arXiv 2602.15293 ↗First posted 17 Feb 2026cs.LG cs.AI cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. In this case, we argue that the natural geometry is information geometry. Our focus is on the role of information geometry on semantic encoding and the linear representation hypothesis. As an illustrative application, we develop "dual steering", a method for robustly steering representations to exhibit a particular concept using linear probes. We prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. Empirically, we find that dual steering enhances the controllability and stability of concept manipulation.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 17 Feb 2026.

Builds on

56 of 66 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2025Gemma 3 Technical Reportcited by 1,686
  8. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  9. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  11. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  12. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
38 more
  1. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  2. 2022Toy Models of Superpositioncited by 928
  3. 2023Steering Language Models With Activation Engineeringcited by 824
  4. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  5. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  6. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 313
  7. 2023Language Models Represent Space and Timecited by 306
  8. 2023In-Context Learning Creates Task Vectorscited by 302
  9. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  10. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  11. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  12. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  13. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  14. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  15. 2025Open Problems in Mechanistic Interpretabilitycited by 173
  16. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  17. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  18. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  19. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 151
  20. 2022Linearly Mapping from Image to Text Spacecited by 139
  21. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  22. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  23. 2022Linear Adversarial Concept Erasurecited by 98
  24. 2017The Riemannian Geometry of Deep Generative Modelscited by 81
  25. 2018An elementary introduction to information geometrycited by 76
  26. 2024On the Origins of Linear Representations in Large Language Modelscited by 76
  27. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 75
  28. 2025Meta CLIP 2: A Worldwide Scaling Recipecited by 59
  29. 2024What Makes and Breaks Safety Fine-tuning? A Mechanistic Studycited by 54
  30. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
  31. 2025The Origins of Representation Manifolds in Large Language Modelscited by 46
  32. 2020Geometrically Enriched Latent Spacescited by 45
  33. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 43
  34. 2024Interpreting the Second-Order Effects of Neurons in CLIPcited by 36
  35. 2022Interpreting Neural Networks through the Polytope Lenscited by 33
  36. 2024Towards Reliable Evaluation of Behavior Steering Interventions in LLMscited by 33
  37. 2020Exploring the Linear Subspace Hypothesis in Gender Bias Mitigationcited by 25
  38. 2021Pulling back information geometrycited by 23

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
  2. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 22
  3. 2026How Controllable Are Large Language Models? A Unified Evaluation across Behavioral Granularitiescited by 3
  4. 2026FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformerscited by 3
  5. 2026Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifoldscited by 3
  6. 2026Stream separation improves Bregman conditioning in transformerscited by 0
  7. 2026MAxBench: A Multinomial Concept Recovery Benchmarkcited by 0
  8. 2026The Answer-Basin Representation Hypothesis: We Are Not Probing or Steering Conceptscited by 0
  9. 2026The Linear Representation Hypothesis for Vision-Language-Action Modelscited by 0