# Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces

arXiv: https://arxiv.org/abs/2107.01667 · HTML: https://mlsanity.com/paper/2107.01667/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 4 Jul 2021.

## Builds on (most cited first, 15 of 15)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [End-to-End Training of Deep Visuomotor Policies](https://mlsanity.com/paper/1504.00702/) (2015)
- [Learning Invariant Representations for Reinforcement Learning without Reconstruction](https://mlsanity.com/paper/2006.10742/) (2020)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [DeepMDP: Learning Continuous Latent Space Models for Representation Learning](https://mlsanity.com/paper/1906.02736/) (2019)
- [State Representation Learning for Control: An Overview](https://mlsanity.com/paper/1802.04181/) (2018)
- [Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation](https://mlsanity.com/paper/1703.00420/) (2017)
- [Plannable Approximations to MDP Homomorphisms: Equivariance under Actions](https://mlsanity.com/paper/2002.11963/) (2020)
- [Combined Reinforcement Learning via Abstract Representations](https://mlsanity.com/paper/1809.04506/) (2018)
- [Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics](https://mlsanity.com/paper/1901.08651/) (2019)
- [PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations](https://mlsanity.com/paper/1705.09805/) (2017)
- [Unsupervised state representation learning with robotic priors: a robustness benchmark](https://mlsanity.com/paper/1709.05185/) (2017)
- [Low Dimensional State Representation Learning with Reward-shaped Priors](https://mlsanity.com/paper/2007.16044/) (2020)

## Built on by (most cited first, 9 of 9)

- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [Playful Interactions for Representation Learning](https://mlsanity.com/paper/2107.09046/) (2021)
- [Automatic Noise Filtering with Dynamic Sparse Training in Deep Reinforcement Learning](https://mlsanity.com/paper/2302.06548/) (2023)
- [Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies](https://mlsanity.com/paper/2403.15267/) (2024)
- [Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models](https://mlsanity.com/paper/2409.05709/) (2024)
- [HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems](https://mlsanity.com/paper/2501.04538/) (2025)
- [PixelBytes: Catching Unified Representation for Multimodal Generation](https://mlsanity.com/paper/2410.01820/) (2024)
- [Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models](https://mlsanity.com/paper/2412.09942/) (2024)
- [HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systems](https://mlsanity.com/paper/2607.19628/) (2026)

## Abstract (arXiv)

Autonomous robots require high degrees of cognitive and motoric intelligence to come into our everyday life. In non-structured environments and in the presence of uncertainties, such degrees of intelligence are not easy to obtain. Reinforcement learning algorithms have proven to be capable of solving complicated robotics tasks in an end-to-end fashion without any need for hand-crafted features or policies. Especially in the context of robotics, in which the cost of real-world data is usually extremely high, reinforcement learning solutions achieving high sample efficiency are needed. In this paper, we propose a framework combining the learning of a low-dimensional state representation, from high-dimensional observations coming from the robot's raw sensory readings, with the learning of the optimal policy, given the learned state representation. We evaluate our framework in the context of mobile robot navigation in the case of continuous state and action spaces. Moreover, we study the problem of transferring what learned in the simulated virtual environment to the real robot without further retraining using real-world data in the presence of visual and depth distractors, such as lighting changes and moving obstacles.