mlsanityData to 7 Oct 2026

Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions

arXiv 2312.16438 ↗First posted 27 Dec 2023cs.RO cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Anchor-bolt insertion is a peg-in-hole task performed in the construction field for holes in concrete. Efforts have been made to automate this task, but the variable lighting and hole surface conditions, as well as the requirements for short setup and task execution time make the automation challenging. In this study, we introduce a vision and proprioceptive data-driven robot control model for this task that is robust to challenging lighting and hole surface conditions. This model consists of a spatial attention point network (SAP) and a deep reinforcement learning (DRL) policy that are trained jointly end-to-end to control the robot. The model is trained in an offline manner, with a sample-efficient framework designed to reduce training time and minimize the reality gap when transferring the model to the physical world. Through evaluations with an industrial robot performing the task in 12 unknown holes, starting from 16 different initial positions, and under three different lighting conditions (two with misleading shadows), we demonstrate that SAP can generate relevant attention points of the image even in challenging lighting conditions. We also show that the proposed model enables task execution with higher success rate and shorter task completion time than various baselines. Due to the proposed model's high effectiveness even in severe lighting, initial positions, and hole conditions, and the offline training framework's high sample-efficiency and short training time, this approach can be easily applied to construction.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 27 references that are in the map. First posted on arXiv 27 Dec 2023.

Builds on

17 of 27 references.

References · 17

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  4. 2018DeepMind Control Suitecited by 935
  5. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  6. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 268
  7. 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
  8. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  9. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 149
  10. 2019Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewardscited by 58
  11. 2017Deep Reinforcement Learning for High Precision Assembly Taskscited by 37
  12. 2021Unsupervised Learning of Visual 3D Keypoints for Controlcited by 30
5 more
  1. 2020Variable Compliance Control for Robotic Peg-in-Hole Assembly: A Deep Reinforcement Learning Approachcited by 25
  2. 2016Learning to Communicate to Solve Riddles with Deep Distributed Recurrent Q-Networkscited by 21
  3. 2021Contact-Rich Manipulation of a Flexible Object based on Deep Predictive Learning using Vision and Tactilitycited by 9
  4. 2020Models, Pixels, and Rewards: Evaluating Design Trade-offs in Visual Model-Based Reinforcement Learningcited by 7
  5. 2021Spatial Attention Point Network for Deep-learning-based Robust Autonomous Robot Motion Generationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensorscited by 4
  2. 2025Trajectory Entropy Reinforcement Learning for Robust Robot Motor Skill Learningcited by 0