mlsanityData to 7 Oct 2026

End-to-End Training of Deep Visuomotor Policies

arXiv 1504.00702 ↗First posted 2 Apr 2015cs.LG cs.CV cs.ROcited by 847

Not on the map. Why?

Cited by 125 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Policy search methods can allow robots to learn control policies for a wide range of tasks, but practical applications of policy search often require hand-engineered components for perception, state estimation, and low-level control. In this paper, we aim to answer the following question: does training the perception and control systems jointly end-to-end provide better performance than training each component separately? To this end, we develop a method that can be used to learn policies that map raw image observations directly to torques at the robot's motors. The policies are represented by deep convolutional neural networks (CNNs) with 92,000 parameters, and are trained using a partially observed guided policy search method, which transforms policy search into supervised learning, with supervision provided by a simple trajectory-centric reinforcement learning method. We evaluate our method on a range of real-world manipulation tasks that require close coordination between vision and control, such as screwing a cap onto a bottle, and present simulated comparisons to a range of prior policy search methods.

Key facts. Cited by 847 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2016: 1; 2017: 0; 2018: 8; 2019: 13; 2020: 10; 2021: 175; 2022: 142; 2023: 138; 2024: 117; 2025: 117; 2026: 126.

847 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  3. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  4. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  5. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  6. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  7. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  8. 2021Implicit Behavioral Cloningcited by 442
  9. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  10. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  11. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  12. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
38 more
  1. 2022Masked Visual Pre-training for Motor Controlcited by 246
  2. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  3. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  4. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  5. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  6. 2022U-NO: U-shaped Neural Operatorscited by 219
  7. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  10. 2021Reinforcement Learning with Prototypical Representationscited by 179
  11. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  12. 2023Language-Driven Representation Learning for Roboticscited by 163
  13. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 155
  14. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  15. 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
  16. 2023Structured World Models from Human Videoscited by 144
  17. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  18. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  19. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  20. 2021SOON: Scenario Oriented Object Navigation with Graph-based Explorationcited by 118
  21. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 116
  22. 2021Learning to drive from a world on railscited by 114
  23. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  24. 2022Reinforcement Learning with Action-Free Pre-Training from Videoscited by 107
  25. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 101
  26. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  27. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  28. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 98
  29. 2021Independent Policy Gradient Methods for Competitive Reinforcement Learningcited by 97
  30. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  31. 2020Offline Meta-Reinforcement Learning with Advantage Weightingcited by 92
  32. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  33. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  34. 2022You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstrationcited by 79
  35. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  36. 2022Auto-Lambda: Disentangling Dynamic Task Relationshipscited by 73
  37. 2023ChessGPT: Bridging Policy Learning and Language Modelingcited by 73
  38. 2024The Ingredients for Robotic Diffusion Transformerscited by 72

50 most cited of 847 All 847 (JSON)