# Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space

arXiv: https://arxiv.org/abs/2605.12412 · HTML: https://mlsanity.com/paper/2605.12412/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 79 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 27 of 27)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization](https://mlsanity.com/paper/2305.19420/) (2023)
- [Automatically Interpreting Millions of Features in Large Language Models](https://mlsanity.com/paper/2410.13928/) (2024)
- [In-Context Learning through the Bayesian Prism](https://mlsanity.com/paper/2306.04891/) (2023)
- [Competition Dynamics Shape Algorithmic Phases of In-Context Learning](https://mlsanity.com/paper/2412.01003/) (2024)
- [Distinguishing the Knowable from the Unknowable with Language Models](https://mlsanity.com/paper/2402.03563/) (2024)
- [Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought](https://mlsanity.com/paper/2603.05488/) (2026)
- [Language Models use Lookbacks to Track Beliefs](https://mlsanity.com/paper/2505.14685/) (2025)
- [The broader spectrum of in-context learning](https://mlsanity.com/paper/2412.03782/) (2024)
- [Belief Dynamics Reveal the Dual Nature of In-Context Learning and Activation Steering](https://mlsanity.com/paper/2511.00617/) (2025)
- [Accumulating Context Changes the Beliefs of Language Models](https://mlsanity.com/paper/2511.01805/) (2025)
- [Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behavior](https://mlsanity.com/paper/2605.05115/) (2026)
- [Priors in Time: Missing Inductive Biases for Language Model Interpretability](https://mlsanity.com/paper/2511.01836/) (2025)
- [Machine Psychology](https://mlsanity.com/paper/2303.13988/) (2023)
- [Bayesian scaling laws for in-context learning](https://mlsanity.com/paper/2410.16531/) (2024)
- [In-Context Learning Strategies Emerge Rationally](https://mlsanity.com/paper/2506.17859/) (2025)
- [Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control](https://mlsanity.com/paper/2604.03147/) (2026)
- [Parameterized Synthetic Text Generation with SimpleStories](https://mlsanity.com/paper/2504.09184/) (2025)
- [Emergence of Hierarchical Emotion Organization in Large Language Models](https://mlsanity.com/paper/2507.10599/) (2025)
- [Context Structure Reshapes the Representational Geometry of Language Models](https://mlsanity.com/paper/2601.22364/) (2026)
- [From Rational Answers to Emotional Resonance: The Role of Controllable Emotion Generation in Language Models](https://mlsanity.com/paper/2502.04075/) (2025)

## Built on by (most cited first, 3 of 3)

- [Planting a Latent Variable in Natural-Looking Text: a More Realistic Test of Belief States in LLMs and Their Link to Concept Geometry](https://mlsanity.com/paper/2608.26887/) (2026)
- [Contextual trajectory and incremental contextual displacement: Towards using LLMs to understand dynamic, utterance-specific meaning construction](https://mlsanity.com/paper/2610.00840/) (2026)
- [The Dichotomy Between Pattern Recognition and Step-by-Step Reasoning](https://mlsanity.com/paper/2610.09186/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) update their behavior in context, which can be viewed as a form of Bayesian inference. However, the structure of the latent hypothesis space over which this inference operates remains unclear. In this work, we propose that LLMs assign beliefs over a low-dimensional geometric space - a conceptual belief space - and that in-context learning corresponds to a trajectory through this space as beliefs are updated over time. Using story understanding as a natural setting for dynamic belief updating, we combine behavioral and representational analyses to study these trajectories. We find that (1) belief updates are well-described as trajectories on low-dimensional, structured manifolds; (2) this structure is reflected consistently in both model behavior and internal representations and can be decoded with simple linear probes to predict behavior; and (3) interventions on these representations causally steer belief trajectories, with effects that can be predicted from the geometry of the conceptual space. Together, our results provide a geometric account of belief dynamics in LLMs, grounding Bayesian interpretations of in-context learning in structured conceptual representations.