# Exploring the Stratified Space Structure of an RL Game with the Volume Growth Transform

arXiv: https://arxiv.org/abs/2507.22010 · HTML: https://mlsanity.com/paper/2507.22010/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 7 of 7)

- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents](https://mlsanity.com/paper/2309.17207/) (2023)
- [Token embeddings violate the manifold hypothesis](https://mlsanity.com/paper/2504.01002/) (2025)
- [Topology of Word Embeddings: Singularities Reflect Polysemy](https://mlsanity.com/paper/2011.09413/) (2020)
- [Topological Singularity Detection at Multiple Scales](https://mlsanity.com/paper/2210.00069/) (2022)
- [The structure of the token space for large language models](https://mlsanity.com/paper/2410.08993/) (2024)

## Built on by (most cited first, 1 of 1)

- [Stratifying Reinforcement Learning with Signal Temporal Logic](https://mlsanity.com/paper/2604.04923/) (2026)

## Abstract (arXiv)

In this work, we explore the structure of the embedding space of a transformer model trained for playing a particular reinforcement learning (RL) game. Specifically, we investigate how a transformer-based Proximal Policy Optimization (PPO) model embeds visual inputs in a simple environment where an agent must collect "coins" while avoiding dynamic obstacles consisting of "spotlights." By adapting Robinson et al.'s study of the volume growth transform for LLMs to the RL setting, we find that the token embedding space for our visual coin collecting game is also not a manifold, and is better modeled as a stratified space, where local dimension can vary from point to point. We further strengthen Robinson's method by proving that fairly general volume growth curves can be realized by stratified spaces. Finally, we carry out an analysis that suggests that as an RL agent acts, its latent representation alternates between periods of low local dimension, while following a fixed sub-strategy, and bursts of high local dimension, where the agent achieves a sub-goal (e.g., collecting an object) or where the environmental complexity increases (e.g., more obstacles appear). Consequently, our work suggests that the distribution of dimensions in a stratified latent space may provide a new geometric indicator of complexity for RL games.