# Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics

arXiv: https://arxiv.org/abs/1901.08651 · HTML: https://mlsanity.com/paper/1901.08651/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2019.

## Built on by (most cited first, 21 of 21)

- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero](https://mlsanity.com/paper/2310.16410/) (2023)
- [Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2408.09675/) (2024)
- [Explainable Reinforcement Learning via a Causal World Model](https://mlsanity.com/paper/2305.02749/) (2023)
- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [Continual Learning in Deep Networks: an Analysis of the Last Layer](https://mlsanity.com/paper/2106.01834/) (2021)
- [Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces](https://mlsanity.com/paper/2107.01667/) (2021)
- [VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts](https://mlsanity.com/paper/2103.07292/) (2021)
- [Demystifying the Mechanisms Behind Emergent Exploration in Goal-conditioned RL](https://mlsanity.com/paper/2510.14129/) (2025)
- [Learning from learning machines: a new generation of AI technology to meet the needs of science](https://mlsanity.com/paper/2111.13786/) (2021)
- [Using Deep Reinforcement Learning with Automatic Curriculum Learning for Mapless Navigation in Intralogistics](https://mlsanity.com/paper/2202.11512/) (2022)
- [Scope and Sense of Explainability for AI-Systems](https://mlsanity.com/paper/2112.10551/) (2021)
- [End-to-end Reinforcement Learning of Robotic Manipulation with Robust Keypoints Representation](https://mlsanity.com/paper/2202.06027/) (2022)
- [Explainable Reinforcement Learning on Financial Stock Trading using SHAP](https://mlsanity.com/paper/2208.08790/) (2022)
- [Kinova Gemini: Interactive Robot Grasping with Visual Reasoning and Conversational AI](https://mlsanity.com/paper/2209.01319/) (2022)
- [Algebras of actions in an agent's representations of the world](https://mlsanity.com/paper/2310.01536/) (2023)
- [Neural Distillation as a State Representation Bottleneck in Reinforcement Learning](https://mlsanity.com/paper/2210.02224/) (2022)
- [POAR: Efficient Policy Optimization via Online Abstract State Representation Learning](https://mlsanity.com/paper/2109.08642/) (2021)
- [CIER: A Novel Experience Replay Approach with Causal Inference in Deep Reinforcement Learning](https://mlsanity.com/paper/2405.08380/) (2024)
- [Recurrent Auto-Encoders for Enhanced Deep Reinforcement Learning in Wilderness Search and Rescue Planning](https://mlsanity.com/paper/2502.19356/) (2025)

## Abstract (arXiv)

Scaling end-to-end reinforcement learning to control real robots from vision presents a series of challenges, in particular in terms of sample efficiency. Against end-to-end learning, state representation learning can help learn a compact, efficient and relevant representation of states that speeds up policy learning, reducing the number of samples needed, and that is easier to interpret. We evaluate several state representation learning methods on goal based robotics tasks and propose a new unsupervised model that stacks representations and combines strengths of several of these approaches. This method encodes all the relevant features, performs on par or better than end-to-end learning with better sample efficiency, and is robust to hyper-parameters change.