# InsertionNet -- A Scalable Solution for Insertion

arXiv: https://arxiv.org/abs/2104.14223 · HTML: https://mlsanity.com/paper/2104.14223/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation](https://mlsanity.com/paper/1806.10293/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields](https://mlsanity.com/paper/1502.07411/) (2015)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1910.05396/) (2019)
- [Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Tasks](https://mlsanity.com/paper/1906.08880/) (2019)
- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [Deep Reinforcement Learning for High Precision Assembly Tasks](https://mlsanity.com/paper/1708.04033/) (2017)
- [A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning](https://mlsanity.com/paper/1810.01531/) (2018)
- [Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning](https://mlsanity.com/paper/2005.10872/) (2020)
- [Compare Contact Model-based Control and Contact Model-free Learning: A Survey of Robotic Peg-in-hole Assembly Strategies](https://mlsanity.com/paper/1904.05240/) (2019)
- [Deep Reinforcement Learning for Contact-Rich Skills Using Compliant Movement Primitives](https://mlsanity.com/paper/2008.13223/) (2020)
- [Improved Exploration through Latent Trajectory Optimization in Deep Deterministic Policy Gradient](https://mlsanity.com/paper/1911.06833/) (2019)

## Built on by (most cited first, 16 of 16)

- [SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning](https://mlsanity.com/paper/2401.16013/) (2024)
- [From Imitation to Refinement -- Residual RL for Precise Assembly](https://mlsanity.com/paper/2407.16677/) (2024)
- [Offline Meta-Reinforcement Learning for Industrial Insertion](https://mlsanity.com/paper/2110.04276/) (2021)
- [JUICER: Data-Efficient Imitation Learning for Robotic Assembly](https://mlsanity.com/paper/2404.03729/) (2024)
- [TacDiffusion: Force-domain Diffusion Policy for Precise Tactile Manipulation](https://mlsanity.com/paper/2409.11047/) (2024)
- [Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision](https://mlsanity.com/paper/2210.15206/) (2022)
- [InsertionNet 2.0: Minimal Contact Multi-Step Insertion Using Multimodal Multiview Sensory Input](https://mlsanity.com/paper/2203.01153/) (2022)
- [MResT: Multi-Resolution Sensing for Real-Time Control with Vision-Language Models](https://mlsanity.com/paper/2401.14502/) (2024)
- [Efficient Skill Acquisition for Complex Manipulation Tasks in Obstructed Environments](https://mlsanity.com/paper/2303.03365/) (2023)
- [AugInsert: Learning Robust Visual-Force Policies via Data Augmentation for Object Assembly Tasks](https://mlsanity.com/paper/2410.14968/) (2024)
- [Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policies](https://mlsanity.com/paper/2507.06519/) (2025)
- [PolyFit: A Peg-in-hole Assembly Framework for Unseen Polygon Shapes via Sim-to-real Adaptation](https://mlsanity.com/paper/2312.02531/) (2023)
- [Automating Deformable Gasket Assembly](https://mlsanity.com/paper/2408.12593/) (2024)
- [Working Backwards: Learning to Place by Picking](https://mlsanity.com/paper/2312.02352/) (2023)
- [A comparison of visual representations for real-world reinforcement learning in the context of vacuum gripping](https://mlsanity.com/paper/2503.02405/) (2025)
- [Proprioception-Anchored Cross-Modal Pretraining for Zero-Shot Sim-to-Real Contact-Rich Assembly](https://mlsanity.com/paper/2609.07534/) (2026)

## Abstract (arXiv)

Complicated assembly processes can be described as a sequence of two main activities: grasping and insertion. While general grasping solutions are common in industry, insertion is still only applicable to small subsets of problems, mainly ones involving simple shapes in fixed locations and in which the variations are not taken into consideration. Recently, RL approaches with prior knowledge (e.g., LfD or residual policy) have been adopted. However, these approaches might be problematic in contact-rich tasks since interaction might endanger the robot and its equipment. In this paper, we tackled this challenge by formulating the problem as a regression problem. By combining visual and force inputs, we demonstrate that our method can scale to 16 different insertion tasks in less than 10 minutes. The resulting policies are robust to changes in the socket position, orientation or peg color, as well as to small differences in peg shape. Finally, we demonstrate an end-to-end solution for 2 complex assembly tasks with multi-insertion objectives when the assembly board is randomly placed on a table.