mlsanityData to 7 Oct 2026

Deep Spatial Autoencoders for Visuomotor Learning

arXiv 1509.06113 ↗First posted 21 Sep 2015cs.LG cs.CV cs.ROcited by 149

Not on the map. Why?

Cited by 30 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning provides a powerful and flexible framework for automated acquisition of robotic motion skills. However, applying reinforcement learning requires a sufficiently detailed representation of the state, including the configuration of task-relevant objects. We present an approach that automates state-space construction by learning a state representation directly from camera images. Our method uses a deep spatial autoencoder to acquire a set of feature points that describe the environment for the current task, such as the positions of objects, and then learns a motion skill with these feature points using an efficient reinforcement learning method based on local linear models. The resulting controller reacts continuously to the learned feature points, allowing the robot to dynamically manipulate objects in the world with closed-loop control. We demonstrate our method with a PR2 robot on tasks that include pushing a free-standing toy block, picking up a bag of rice using a spatula, and hanging a loop of rope on a hook at various positions. In each task, our method automatically learns to track task-relevant objects and manipulate their configuration with the robot's arm.

Key facts. Cited by 149 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Sep 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 2; 2021: 41; 2022: 38; 2023: 20; 2024: 23; 2025: 7; 2026: 16.

149 papers.

Sort:

By area:

  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  2. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  4. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  5. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  6. 2022Masked World Models for Visual Controlcited by 169
  7. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 135
  8. 2021Pathdreamer: A World Model for Indoor Navigationcited by 117
  9. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  10. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  11. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  12. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
38 more
  1. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  2. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Modelscited by 77
  3. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  4. 2022How to Leverage Unlabeled Data in Offline Reinforcement Learningcited by 61
  5. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 60
  6. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  7. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  8. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 49
  9. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  10. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 46
  11. 2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
  12. 2021Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularizationcited by 44
  13. 2022Reinforcement Learning with Neural Radiance Fieldscited by 43
  14. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
  15. 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 42
  16. 2025Towards General-Purpose Model-Free Reinforcement Learningcited by 40
  17. 2024JUICER: Data-Efficient Imitation Learning for Robotic Assemblycited by 36
  18. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  19. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  20. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 32
  21. 2023Inverse Dynamics Pretraining Learns Good Representations for Multitask Imitationcited by 30
  22. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 29
  23. 2021Which Mutual-Information Representation Learning Objectives are Sufficient for Control?cited by 28
  24. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  25. 2021Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Controlcited by 24
  26. 2022Investigating the Properties of Neural Network Representations in Reinforcement Learningcited by 24
  27. 2023Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfercited by 23
  28. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replaycited by 21
  29. 2022Goal-conditioned dual-action imitation learning for dexterous dual-arm robot manipulationcited by 21
  30. 2022Visuomotor Control in Multi-Object Scenes Using Object-Aware Representationscited by 21
  31. 2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 21
  32. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 20
  33. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 19
  34. 2022Unsupervised Image Representation Learning with Deep Latent Particlescited by 19
  35. 2022Back to the Manifold: Recovering from Out-of-Distribution Statescited by 19
  36. 2023Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Datacited by 19
  37. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  38. 2023Distance Weighted Supervised Learning for Offline Interaction Datacited by 18

50 most cited of 149 All 149 (JSON)