mlsanityData to 14 Oct 2026

A Visual Reinforcement Learning-Based Separate Primitive Policy for Peg-in-Hole Tasks

arXiv 2504.14820 ↗First posted 21 Apr 2025cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

For peg-in-hole tasks, humans rely on binocular visual perception to locate the peg above the hole surface and then proceed with insertion. This paper draws insights from this behavior to enable agents to learn efficient assembly strategies through visual reinforcement learning. Hence, we propose a Separate Primitive Policy (S2P) to learn how to derive location and insertion actions simultaneously. S2P is compatible with model-free reinforcement learning algorithms. Ten insertion tasks featuring different polygons are developed as benchmarks for evaluations. Simulation experiments show that S2P can boost the sample efficiency and success rate even with force constraints. Real-world experiments are also performed to verify the feasibility of S2P. Ablations are finally given to discuss the generalizability of S2P and some factors that affect its performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 42 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

31 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning22 Jul
  2. 2024A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning27 May

Other references · 29

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  4. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  5. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  6. 2020Reinforcement Learning with Augmented Datacited by 468
  7. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  8. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
  9. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  10. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 121
  11. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  12. 2021Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Taskscited by 78
17 more
  1. 2022Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learningcited by 60
  2. 2023TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learningcited by 57
  3. 2022On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baselinecited by 56
  4. 2021Tactile-RL for Insertion: Generalization to Objects of Unknown Geometrycited by 48
  5. 2019Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assemblycited by 39
  6. 2022Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learningcited by 34
  7. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 28
  8. 2024TacSL: A Library for Visuotactile Sensor Simulation and Learningcited by 27
  9. 2023The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learningcited by 26
  10. 2024AutoMate: Specialist and Generalist Assembly Policies over Diverse Geometriescited by 26
  11. 2021Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Taskscited by 20
  12. 2021Towards Hierarchical Task Decomposition using Deep Reinforcement Learning for Pick and Place Subtaskscited by 14
  13. 2020Learning Sequences of Manipulation Primitives for Robotic Assemblycited by 6
  14. 2023Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditionscited by 5
  15. 2020Proactive Action Visual Residual Reinforcement Learning for Contact-Rich Tasks Using a Torque-Controlled Robotcited by 4
  16. 2021Hierarchical Primitive Composition: Simultaneous Activation of Skills with Inconsistent Action Dimensions in Multiple Hierarchiescited by 3
  17. 2023PolyFit: A Peg-in-hole Assembly Framework for Unseen Polygon Shapes via Sim-to-real Adaptationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Visual-Tactile Peg-in-Hole Assembly Learning from Peg-out-of-Hole Disassembly ↗
  2. 2025A General Peg-in-Hole Assembly Policy Based on Domain Randomized Reinforcement Learning
  3. 2019Compare Contact Model-based Control and Contact Model-free Learning: A Survey of Robotic Peg-in-hole Assembly Strategies
  4. 2023Evaluating Robustness of Visual Representations for Object Assembly Task Requiring Spatio-Geometrical Reasoning
  5. 2020Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Tasks
  6. 2025Zero-Shot Peg Insertion: Identifying Mating Holes and Estimating SE(2) Poses with Vision-Language Models
  7. 2024Symmetry-aware Reinforcement Learning for Robotic Assembly under Partial Observability with a Soft Wrist
  8. 2024Leveraging Procedural Generation for Learning Autonomous Peg-in-Hole Assembly in Space