# Self-supervised Learning of Image Embedding for Continuous Control

arXiv: https://arxiv.org/abs/1901.00943 · HTML: https://mlsanity.com/paper/1901.00943/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2019.

## Built on by (most cited first, 16 of 16)

- [Contrastive Learning as Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2206.07568/) (2022)
- [Language Conditioned Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2005.07648/) (2020)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [Discovering and Achieving Goals via World Models](https://mlsanity.com/paper/2110.09514/) (2021)
- [METRA: Scalable Unsupervised RL with Metric-Aware Abstraction](https://mlsanity.com/paper/2310.08887/) (2023)
- [Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data](https://mlsanity.com/paper/2306.03346/) (2023)
- [Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning](https://mlsanity.com/paper/2104.05043/) (2021)
- [A Survey of State Representation Learning for Deep Reinforcement Learning](https://mlsanity.com/paper/2506.17518/) (2025)
- [Visual Affordance Prediction for Guiding Robot Exploration](https://mlsanity.com/paper/2305.17783/) (2023)
- [Wish you were here: Hindsight Goal Selection for long-horizon dexterous manipulation](https://mlsanity.com/paper/2112.00597/) (2021)
- [Adjacency constraint for efficient hierarchical reinforcement learning](https://mlsanity.com/paper/2111.00213/) (2021)
- [Learning Goal-Conditioned Representations for Language Reward Models](https://mlsanity.com/paper/2407.13887/) (2024)
- [Case-Based Inverse Reinforcement Learning Using Temporal Coherence](https://mlsanity.com/paper/2206.05827/) (2022)
- [Periodic Skill Discovery](https://mlsanity.com/paper/2511.03187/) (2025)
- [Geometry of Uncertainty: Learning Metric Spaces for Multimodal State Estimation in RL](https://mlsanity.com/paper/2602.12087/) (2026)
- [ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning](https://mlsanity.com/paper/2609.36238/) (2026)

## Abstract (arXiv)

Operating directly from raw high dimensional sensory inputs like images is still a challenge for robotic control. Recently, Reinforcement Learning methods have been proposed to solve specific tasks end-to-end, from pixels to torques. However, these approaches assume the access to a specified reward which may require specialized instrumentation of the environment. Furthermore, the obtained policy and representations tend to be task specific and may not transfer well. In this work we investigate completely self-supervised learning of a general image embedding and control primitives, based on finding the shortest time to reach any state. We also introduce a new structure for the state-action value function that builds a connection between model-free and model-based methods, and improves the performance of the learning algorithm. We experimentally demonstrate these findings in three simulated robotic tasks.