mlsanityData to 14 Oct 2026

Vision-Based Multi-Task Manipulation for Inexpensive Robots Using End-To-End Learning from Demonstration

arXiv 1707.02920 ↗First posted 10 Jul 2017cs.LG cs.AI cs.ROcited by 100

Not on the map. Why?

Cited by 53 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a technique for multi-task learning from demonstration that trains the controller of a low-cost robotic arm to accomplish several complex picking and placing tasks, as well as non-prehensile manipulation. The controller is a recurrent neural network using raw images as input and generating robot arm trajectories, with the parameters shared across the tasks. The controller also combines VAE-GAN-based reconstruction with autoregressive multimodal action prediction. Our results demonstrate that it is possible to learn complex manipulation tasks, such as picking up a towel, wiping an object, and depositing the towel to its previous position, entirely from raw images with direct behavior cloning. We show that weight sharing and reconstruction-based regularization substantially improve generalization and robustness, and training on multiple tasks simultaneously increases the success rate on all tasks.

Key facts. Cited by 100 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Jul 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 3; 2019: 4; 2020: 6; 2021: 10; 2022: 17; 2023: 21; 2024: 16; 2025: 14; 2026: 9.

100 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  2. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  3. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  4. 2021Implicit Behavioral Cloningcited by 452
  5. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  6. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  7. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  8. 2022Interactive Language: Talking to Robots in Real Timecited by 218
  9. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  10. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 183
  11. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 169
  12. 2023Getting aligned on representational alignmentcited by 143
38 more
  1. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 138
  2. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  3. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  4. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  5. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  6. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  7. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 93
  8. 2023XSkill: Cross Embodiment Skill Discoverycited by 75
  9. 2024Unpacking Failure Modes of Generative Policies: Runtime Monitoring of Consistency and Progresscited by 69
  10. 2024Equivariant Diffusion Policycited by 66
  11. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 61
  12. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 57
  13. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  14. 2025Failure Prediction at Runtime for Generative Robot Policiescited by 42
  15. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 40
  16. 2024Surgical Robot Transformer (SRT): Imitation Learning for Surgical Taskscited by 40
  17. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Taskscited by 33
  18. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  19. 2023CCIL: Continuity-based Data Augmentation for Corrective Imitation Learningcited by 29
  20. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewardscited by 28
  21. 2020Semi-supervised reward learning for offline reinforcement learningcited by 23
  22. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  23. 2024DNAct: Diffusion Guided Multi-Task 3D Policy Learningcited by 23
  24. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  25. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replaycited by 21
  26. 2024Don't Start from Scratch: Behavioral Refinement via Interpolant-based Policy Diffusioncited by 21
  27. 2019Scaling data-driven robotics with reward sketching and batch reinforcement learningcited by 18
  28. 2023Chain-of-Thought Predictive Controlcited by 18
  29. 2023Transferring Foundation Models for Generalizable Robotic Manipulationcited by 18
  30. 2023Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flightcited by 15
  31. 2022Bayesian Imitation Learning for End-to-End Mobile Manipulationcited by 14
  32. 2022Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion Speedcited by 14
  33. 2024Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulationcited by 14
  34. 2025A Survey on Imitation Learning for Contact-Rich Tasks in Roboticscited by 14
  35. 2025IMLE Policy: Fast and Sample Efficient Visuomotor Policy Learning via Implicit Maximum Likelihood Estimationcited by 13
  36. 2025FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistencycited by 13
  37. 2022Continual Learning from Demonstration of Robotics Skillscited by 12
  38. 2020Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learningcited by 11

50 most cited of 100 All 100 (JSON)