# Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks

arXiv: https://arxiv.org/abs/2110.12618 · HTML: https://mlsanity.com/paper/2110.12618/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2021.

## Built on by (most cited first, 20 of 20)

- [From Imitation to Refinement -- Residual RL for Precise Assembly](https://mlsanity.com/paper/2407.16677/) (2024)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [JUICER: Data-Efficient Imitation Learning for Robotic Assembly](https://mlsanity.com/paper/2404.03729/) (2024)
- [A Survey of Embodied Learning for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2408.11537/) (2024)
- [Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulation](https://mlsanity.com/paper/2412.02676/) (2024)
- [Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection](https://mlsanity.com/paper/2312.10571/) (2023)
- [DRL-Based Optimization for AoI and Energy Consumption in C-V2X Enabled IoV](https://mlsanity.com/paper/2411.13104/) (2024)
- [Learning Excavation of Rigid Objects with Offline Reinforcement Learning](https://mlsanity.com/paper/2303.16427/) (2023)
- [Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policies](https://mlsanity.com/paper/2507.06519/) (2025)
- [Diff-Transfer: Model-based Robotic Manipulation Skill Transfer via Differentiable Physics Simulation](https://mlsanity.com/paper/2310.04930/) (2023)
- [Automating Deformable Gasket Assembly](https://mlsanity.com/paper/2408.12593/) (2024)
- [Leveraging Extrinsic Dexterity for Occluded Grasping on Grasp Constraining Walls](https://mlsanity.com/paper/2507.14721/) (2025)
- [Hybrid TD3: Overestimation Bias Analysis and Stable Policy Optimization for Hybrid Action Space](https://mlsanity.com/paper/2603.01302/) (2026)
- [Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes](https://mlsanity.com/paper/2205.04297/) (2022)
- [Learn A Flexible Exploration Model for Parameterized Action Markov Decision Processes](https://mlsanity.com/paper/2501.02774/) (2025)
- [Hybrid Robot Learning for Automatic Robot Motion Planning in Manufacturing](https://mlsanity.com/paper/2502.19340/) (2025)
- [A Visual Reinforcement Learning-Based Separate Primitive Policy for Peg-in-Hole Tasks](https://mlsanity.com/paper/2504.14820/) (2025)
- [SI-Diff: A Framework for Learning Search and High-Precision Insertion with a Force-Domain Diffusion Policy](https://mlsanity.com/paper/2605.12247/) (2026)
- [HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models](https://mlsanity.com/paper/2606.27627/) (2026)
- [Chronos Policy: Learning Robot Policies on Physical Time](https://mlsanity.com/paper/2610.13929/) (2026)

## Abstract (arXiv)

This paper introduces a discrete-continuous action space to learn insertion primitives for robotic assembly tasks. Primitive is a sequence of elementary actions with certain exit conditions, such as "pushing down the peg until contact". Since the primitive is an abstraction of robot control commands and encodes human prior knowledge, it reduces the exploration difficulty and yields better learning efficiency. In this paper, we learn robot assembly skills via primitives. Specifically, we formulate insertion primitives as parameterized actions: hybrid actions consisting of discrete primitive types and continuous primitive parameters. Compared with the previous work using a set of discretized parameters for each primitive, the agent in our method can freely choose primitive parameters from a continuous space, which is more flexible and efficient. To learn these insertion primitives, we propose Twin-Smoothed Multi-pass Deep Q-Network (TS-MP-DQN), an advanced version of MP-DQN with twin Q-network to reduce the Q-value over-estimation. Extensive experiments are conducted in the simulation and real world for validation. From experiment results, our approach achieves higher success rates than three baselines: MP-DQN with parameterized actions, primitives with discrete parameters, and continuous velocity control. Furthermore, learned primitives are robust to sim-to-real transfer and can generalize to challenging assembly tasks such as tight round peg-hole and complex shaped electric connectors with promising success rates. Experiment videos are available at https://msc.berkeley.edu/research/insertion-primitives.html.