# Using PCA to Efficiently Represent State Spaces

arXiv: https://arxiv.org/abs/1505.00322 · HTML: https://mlsanity.com/paper/1505.00322/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 May 2015.

## Built on by (most cited first, 5 of 5)

- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [State Representation Learning from Demonstration](https://mlsanity.com/paper/1910.01738/) (2019)
- [A novel multi-agent dynamic portfolio optimization learning system based on hierarchical deep reinforcement learning](https://mlsanity.com/paper/2501.06832/) (2025)
- [Alert: Learning Trigger Functions for Early Classification of Time Series using Deep-RL](https://mlsanity.com/paper/2502.06584/) (2025)
- [Deep Spectral Q-learning with Application to Mobile Health](https://mlsanity.com/paper/2301.00927/) (2023)

## Abstract (arXiv)

Reinforcement learning algorithms need to deal with the exponential growth of states and actions when exploring optimal control in high-dimensional spaces. This is known as the curse of dimensionality. By projecting the agent's state onto a low-dimensional manifold, we can represent the state space in a smaller and more efficient representation. By using this representation during learning, the agent can converge to a good policy much faster. We test this approach in the Mario Benchmarking Domain. When using dimensionality reduction in Mario, learning converges much faster to a good policy. But, there is a critical convergence-performance trade-off. By projecting onto a low-dimensional manifold, we are ignoring important data. In this paper, we explore this trade-off of convergence and performance. We find that learning in as few as 4 dimensions (instead of 9), we can improve performance past learning in the full dimensional space at a faster convergence rate.