# State Representation Learning for Control: An Overview

arXiv: https://arxiv.org/abs/1802.04181 · HTML: https://mlsanity.com/paper/1802.04181/

## Key facts

- Cited by 127 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Feb 2018.

## Built on by (most cited first, 30 of 127)

- [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023)
- [Pretraining Representations for Data-Efficient Reinforcement Learning](https://mlsanity.com/paper/2106.04799/) (2021)
- [TesserAct: Learning 4D Embodied World Models](https://mlsanity.com/paper/2504.20995/) (2025)
- [Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models](https://mlsanity.com/paper/2502.14819/) (2025)
- [Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos](https://mlsanity.com/paper/2101.07241/) (2021)
- [Challenges and Opportunities in Offline Reinforcement Learning from Visual Observations](https://mlsanity.com/paper/2206.04779/) (2022)
- [Self-Supervised Multimodal Learning: A Survey](https://mlsanity.com/paper/2304.01008/) (2023)
- [Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning](https://mlsanity.com/paper/2408.14472/) (2024)
- [Bridging State and History Representations: Understanding Self-Predictive RL](https://mlsanity.com/paper/2401.08898/) (2024)
- [Reinforcement Learning with Neural Radiance Fields](https://mlsanity.com/paper/2206.01634/) (2022)
- [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021)
- [Training Larger Networks for Deep Reinforcement Learning](https://mlsanity.com/paper/2102.07920/) (2021)
- [BISCUIT: Causal Representation Learning from Binary Interactions](https://mlsanity.com/paper/2306.09643/) (2023)
- [World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challenges](https://mlsanity.com/paper/2301.05832/) (2023)
- [Provable Representation Learning for Imitation with Contrastive Fourier Features](https://mlsanity.com/paper/2105.12272/) (2021)
- [Geometric Entropic Exploration](https://mlsanity.com/paper/2101.02055/) (2021)
- [Action-Sufficient State Representation Learning for Control with Structural Constraints](https://mlsanity.com/paper/2110.05721/) (2021)
- [Which Mutual-Information Representation Learning Objectives are Sufficient for Control?](https://mlsanity.com/paper/2106.07278/) (2021)
- [JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2112.04907/) (2021)
- [LLM-Empowered State Representation for Reinforcement Learning](https://mlsanity.com/paper/2407.13237/) (2024)
- [World-Gymnast: Training Robots with Reinforcement Learning in a World Model](https://mlsanity.com/paper/2602.02454/) (2026)
- [Investigating the Properties of Neural Network Representations in Reinforcement Learning](https://mlsanity.com/paper/2203.15955/) (2022)
- [Meta-Adaptive Nonlinear Control: Theory and Algorithms](https://mlsanity.com/paper/2106.06098/) (2021)
- [Multimodal VAE Active Inference Controller](https://mlsanity.com/paper/2103.04412/) (2021)
- [Back to the Manifold: Recovering from Out-of-Distribution States](https://mlsanity.com/paper/2207.08673/) (2022)
- [A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation](https://mlsanity.com/paper/2106.06854/) (2021)
- [Concept Learning for Interpretable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.12232/) (2023)
- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictability](https://mlsanity.com/paper/2309.00082/) (2023)
- [Continual Learning in Deep Networks: an Analysis of the Last Layer](https://mlsanity.com/paper/2106.01834/) (2021)

## Abstract (arXiv)

Representation learning algorithms are designed to learn abstract features that characterize data. State representation learning (SRL) focuses on a particular kind of representation learning where learned features are in low dimension, evolve through time, and are influenced by actions of an agent. The representation is learned to capture the variation in the environment generated by the agent's actions; this kind of representation is particularly suitable for robotics and control scenarios. In particular, the low dimension characteristic of the representation helps to overcome the curse of dimensionality, provides easier interpretation and utilization by humans and can help improve performance and speed in policy learning algorithms such as reinforcement learning. This survey aims at covering the state-of-the-art on state representation learning in the most recent years. It reviews different SRL methods that involve interaction with the environment, their implementations and their applications in robotics control tasks (simulated or real). In particular, it highlights how generic learning objectives are differently exploited in the reviewed algorithms. Finally, it discusses evaluation methods to assess the representation learned and summarizes current and future lines of research.