mlsanityData to 7 Oct 2026

State Representation Learning from Demonstration

arXiv 1910.01738 ↗First posted 15 Sep 2019cs.LG cs.RO stat.MLcited by 5

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Robots could learn their own state and world representation from perception and experience without supervision. This desirable goal is the main focus of our field of interest, state representation learning (SRL). Indeed, a compact representation of such a state is beneficial to help robots grasp onto their environment for interacting. The properties of this representation have a strong impact on the adaptive capability of the agent. In this article we present an approach based on imitation learning. The idea is to train several policies that share the same representation to reproduce various demonstrations. To do so, we use a multi-head neural network with a shared state representation feeding a task-specific agent. If the demonstrations are diverse, the trained representation will eventually contain the information necessary for all tasks, while discarding irrelevant information. As such, it will potentially become a compact state representation useful for new tasks. We call this approach SRLfD (State Representation Learning from Demonstration). Our experiments confirm that when a controller takes SRLfD-based representations as input, it can achieve better performance than with other representation strategies and promote more efficient reinforcement learning (RL) than with an end-to-end RL strategy.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 72 references that are in the map. First posted on arXiv 15 Sep 2019.

Builds on

30 of 72 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2016OpenAI Gymcited by 1,684
  8. 2018World Modelscited by 1,444
  9. 2017Hindsight Experience Replaycited by 1,096
  10. 2018DeepMind Control Suitecited by 935
  11. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  12. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 515
18 more
  1. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  2. 2015Policy Distillationcited by 414
  3. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  4. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  5. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 266
  6. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  7. 2016Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learningcited by 209
  8. 2016A guide to convolution arithmetic for deep learningcited by 182
  9. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 149
  10. 2018State Representation Learning for Control: An Overviewcited by 125
  11. 2016Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfercited by 124
  12. 2016Loss is its own Reward: Self-Supervision for Reinforcement Learningcited by 72
  13. 2019Weight Agnostic Neural Networkscited by 71
  14. 2016Learning to Push by Grasping: Using multiple tasks for effective learningcited by 36
  15. 2017PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representationscited by 16
  16. 2015Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Modelscited by 6
  17. 2018S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learningcited by 6
  18. 2015Using PCA to Efficiently Represent State Spacescited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 1.

5 papers.

  1. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  2. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  3. 2023Few-Shot Learning of Force-Based Motions From Demonstration Through Pre-training of Haptic Representationcited by 5
  4. 2021Exploratory State Representation Learningcited by 2
  5. 2022The Pump Scheduling Problem: A Real-World Scenario for Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Domain-Adversarial and Conditional State Space Model for Imitation Learning
  2. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
  3. 2024Enabling Stateful Behaviors for Diffusion-based Policy Learning
  4. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning
  5. 2024Learning with Language-Guided State Abstractions
  6. 2021Manipulator-Independent Representations for Visual Imitation
  7. 2022Multi-Task Imitation Learning for Linear Dynamical Systems
  8. 2022Self-Imitation Learning from Demonstrations