mlsanityData to 7 Oct 2026

Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards

arXiv 1906.05841 ↗First posted 13 Jun 2019cs.RO cs.CV cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

Connector insertion and many other tasks commonly found in modern manufacturing settings involve complex contact dynamics and friction. Since it is difficult to capture related physical effects with first-order modeling, traditional control methods often result in brittle and inaccurate controllers, which have to be manually tuned. Reinforcement learning (RL) methods have been demonstrated to be capable of learning controllers in such environments from autonomous interaction with the environment, but running RL algorithms in the real world poses sample efficiency and safety challenges. Moreover, in practical real-world settings we cannot assume access to perfect state information or dense reward signals. In this paper, we consider a variety of difficult industrial insertion tasks with visual inputs and different natural reward specifications, namely sparse rewards and goal images. We show that methods that combine RL with prior information, such as classical controllers or demonstrations, can solve these tasks from a reasonable amount of real-world interaction.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 18; 2022: 9; 2023: 8; 2024: 10; 2025: 9; 2026: 2.

58 papers.

Sort:

By area:

  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  2. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  3. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  4. 2024SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learningcited by 113
  5. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  6. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  7. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  8. 2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 37
  9. 2023MimicTouch: Leveraging Multi-modal Human Tactile Demonstrations for Contact-rich Manipulationcited by 30
  10. 2021Residual Reinforcement Learning from Demonstrationscited by 27
  11. 2020Residual Learning from Demonstration: Adapting DMPs for Contact-rich Manipulationcited by 22
  12. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 18
38 more
  1. 2021InsertionNet -- A Scalable Solution for Insertioncited by 15
  2. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
  3. 2022Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Visioncited by 11
  4. 2022Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewardscited by 10
  5. 2021LS3: Latent Space Safe Sets for Long-Horizon Visuomotor Control of Sparse Reward Iterative Taskscited by 9
  6. 2021Learning to Centralize Dual-Arm Assemblycited by 9
  7. 2021Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Modelscited by 9
  8. 2022InsertionNet 2.0: Minimal Contact Multi-Step Insertion Using Multimodal Multiview Sensory Inputcited by 9
  9. 2024GenCHiP: Generating Robot Policy Code for High-Precision and Contact-Rich Manipulation Taskscited by 9
  10. 2024MILES: Making Imitation Learning Easy with Self-Supervisioncited by 9
  11. 2020Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Taskscited by 8
  12. 2024Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Graspingcited by 8
  13. 2021Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertaintycited by 6
  14. 2025Accelerating Residual Reinforcement Learning with Uncertainty Estimationcited by 6
  15. 2021Data-Efficient Deep Reinforcement Learning for Attitude Control of Fixed-Wing UAVs: Field Experimentscited by 4
  16. 2023Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learningcited by 4
  17. 2024Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priorscited by 4
  18. 2025Flow-based Domain Randomization for Learning and Sequencing Robotic Skillscited by 4
  19. 2025IMPACT: Intelligent Motion Planning with Acceptable Contact Trajectories via Vision-Language Modelscited by 4
  20. 2026UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Modelscited by 4
  21. 2021A Study on Dense and Sparse (Visual) Rewards in Robot Policy Learningcited by 3
  22. 2021Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems through Probabilistic Model Checkingcited by 3
  23. 2021Back to Reality for Imitation Learningcited by 3
  24. 2022Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertioncited by 3
  25. 2021Adaptive control of a mechatronic system using constrained residual reinforcement learningcited by 2
  26. 2022Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policiescited by 2
  27. 2023Zero-Shot Transfer of Haptics-Based Object Insertion Policiescited by 2
  28. 2023Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditionscited by 2
  29. 2025PANDORA: Diffusion Policy Learning for Dexterous Robotic Piano Playingcited by 2
  30. 2021Combining Learning from Demonstration with Learning by Exploration to Facilitate Contact-Rich Taskscited by 1
  31. 2022Learning Classifier Systems for Self-Explaining Socio-Technical-Systemscited by 1
  32. 2023Learning Vision-based Robotic Manipulation Tasks Sequentially in Offline Reinforcement Learning Settingscited by 1
  33. 2023Learning Human-Inspired Force Strategies for Robotic Assemblycited by 1
  34. 2024Automating Deformable Gasket Assemblycited by 1
  35. 2025CHEQ-ing the Box: Safe Variable Impedance Learning for Robotic Polishingcited by 1
  36. 2021Progressive extension of reinforcement learning action dimension for asymmetric assembly taskscited by 0
  37. 2021A study of first-passage time minimization via Q-learning in heated gridworldscited by 0
  38. 2022Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holescited by 0

50 most cited of 58 All 58 (JSON)