# InsertionNet 2.0: Minimal Contact Multi-Step Insertion Using Multimodal Multiview Sensory Input

arXiv: https://arxiv.org/abs/2203.01153 · HTML: https://mlsanity.com/paper/2203.01153/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [Generalizing from a Few Examples: A Survey on Few-Shot Learning](https://mlsanity.com/paper/1904.05046/) (2019)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Sim-to-Real via Sim-to-Sim: Data-efficient Robotic Grasping via Randomized-to-Canonical Adaptation Networks](https://mlsanity.com/paper/1812.07252/) (2018)
- [Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks](https://mlsanity.com/paper/1810.10191/) (2018)
- [Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewards](https://mlsanity.com/paper/1906.05841/) (2019)
- [Deep Reinforcement Learning for High Precision Assembly Tasks](https://mlsanity.com/paper/1708.04033/) (2017)
- [A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learning](https://mlsanity.com/paper/1810.01531/) (2018)
- [Guided Uncertainty-Aware Policy Optimization: Combining Learning and Model-Based Strategies for Sample-Efficient Policy Learning](https://mlsanity.com/paper/2005.10872/) (2020)
- [InsertionNet -- A Scalable Solution for Insertion](https://mlsanity.com/paper/2104.14223/) (2021)
- [Compare Contact Model-based Control and Contact Model-free Learning: A Survey of Robotic Peg-in-hole Assembly Strategies](https://mlsanity.com/paper/1904.05240/) (2019)
- [Deep Reinforcement Learning for Contact-Rich Skills Using Compliant Movement Primitives](https://mlsanity.com/paper/2008.13223/) (2020)

## Built on by (most cited first, 9 of 9)

- [JUICER: Data-Efficient Imitation Learning for Robotic Assembly](https://mlsanity.com/paper/2404.03729/) (2024)
- [TacDiffusion: Force-domain Diffusion Policy for Precise Tactile Manipulation](https://mlsanity.com/paper/2409.11047/) (2024)
- [Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision](https://mlsanity.com/paper/2210.15206/) (2022)
- [Manual2Skill++: Connector-Aware General Robotic Assembly from Instruction Manuals via Vision-Language Models](https://mlsanity.com/paper/2510.16344/) (2025)
- [AugInsert: Learning Robust Visual-Force Policies via Data Augmentation for Object Assembly Tasks](https://mlsanity.com/paper/2410.14968/) (2024)
- [PolyFit: A Peg-in-hole Assembly Framework for Unseen Polygon Shapes via Sim-to-real Adaptation](https://mlsanity.com/paper/2312.02531/) (2023)
- [Towards Natural Language-Driven Assembly Using Foundation Models](https://mlsanity.com/paper/2406.16093/) (2024)
- [Working Backwards: Learning to Place by Picking](https://mlsanity.com/paper/2312.02352/) (2023)
- [EasyInsert: A Data-Efficient and Generalizable Insertion Policy](https://mlsanity.com/paper/2505.16187/) (2025)

## Abstract (arXiv)

We address the problem of devising the means for a robot to rapidly and safely learn insertion skills with just a few human interventions and without hand-crafted rewards or demonstrations. Our InsertionNet version 2.0 provides an improved technique to robustly cope with a wide range of use-cases featuring different shapes, colors, initial poses, etc. In particular, we present a regression-based method based on multimodal input from stereo perception and force, augmented with contrastive learning for the efficient learning of valuable features. In addition, we introduce a one-shot learning technique for insertion, which relies on a relation network scheme to better exploit the collected data and to support multi-step insertion tasks. Our method improves on the results obtained with the original InsertionNet, achieving an almost perfect score (above 97.5% on 200 trials) in 16 real-life insertion tasks while minimizing the execution time and contact during insertion. We further demonstrate our method's ability to tackle a real-life 3-step insertion task and perfectly solve an unseen insertion task without learning.