# Joint Learning of Instance and Semantic Segmentation for Robotic Pick-and-Place with Heavy Occlusions in Clutter

arXiv: https://arxiv.org/abs/2001.07481 · HTML: https://mlsanity.com/paper/2001.07481/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2020.

## Built on by (most cited first, 12 of 12)

- [Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling](https://mlsanity.com/paper/2109.11103/) (2021)
- [Image Amodal Completion: A Survey](https://mlsanity.com/paper/2207.02062/) (2022)
- [Learning to Better Segment Objects from Unseen Classes with Unlabeled Videos](https://mlsanity.com/paper/2104.12276/) (2021)
- [BLADE: Box-Level Supervised Amodal Segmentation through Directed Expansion](https://mlsanity.com/paper/2401.01642/) (2024)
- [Towards Robust Part-aware Instance Segmentation for Industrial Bin Picking](https://mlsanity.com/paper/2203.02767/) (2022)
- [Smart Explorer: Recognizing Objects in Dense Clutter via Interactive Exploration](https://mlsanity.com/paper/2208.03496/) (2022)
- [PLUG: Revisiting Amodal Segmentation with Foundation Model and Hierarchical Focus](https://mlsanity.com/paper/2405.16094/) (2024)
- [Learn Fast, Segment Well: Fast Object Segmentation Learning on the iCub Robot](https://mlsanity.com/paper/2206.13462/) (2022)
- [Category-level Shape Estimation for Densely Cluttered Objects](https://mlsanity.com/paper/2302.11983/) (2023)
- [Sequential Amodal Segmentation via Cumulative Occlusion Learning](https://mlsanity.com/paper/2405.05791/) (2024)
- [SKU-Patch: Towards Efficient Instance Segmentation for Unseen Objects in Auto-Store](https://mlsanity.com/paper/2311.04645/) (2023)
- [Amodal Segmentation for Laparoscopic Surgery Video Instruments](https://mlsanity.com/paper/2408.01067/) (2024)

## Abstract (arXiv)

We present joint learning of instance and semantic segmentation for visible and occluded region masks. Sharing the feature extractor with instance occlusion segmentation, we introduce semantic occlusion segmentation into the instance segmentation model. This joint learning fuses the instance- and image-level reasoning of the mask prediction on the different segmentation tasks, which was missing in the previous work of learning instance segmentation only (instance-only). In the experiments, we evaluated the proposed joint learning comparing the instance-only learning on the test dataset. We also applied the joint learning model to 2 different types of robotic pick-and-place tasks (random and target picking) and evaluated its effectiveness to achieve real-world robotic tasks.