# Learning to Segment Human Body Parts with Synthetically Trained Deep Convolutional Networks

arXiv: https://arxiv.org/abs/2102.01460 · HTML: https://mlsanity.com/paper/2102.01460/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 2 Feb 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation](https://mlsanity.com/paper/1707.03718/) (2017)
- [Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?](https://mlsanity.com/paper/1610.01983/) (2016)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Pyramid Attention Network for Semantic Segmentation](https://mlsanity.com/paper/1805.10180/) (2018)
- [Dual Path Networks](https://mlsanity.com/paper/1707.01629/) (2017)
- [On Pre-Trained Image Features and Synthetic Images for Deep Learning](https://mlsanity.com/paper/1710.10710/) (2017)
- [Learning Multi-scale Features for Foreground Segmentation](https://mlsanity.com/paper/1808.01477/) (2018)
- [Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection](https://mlsanity.com/paper/1612.03019/) (2016)
- [Building an Aerial-Ground Robotics System for Precision Farming: An Adaptable Solution](https://mlsanity.com/paper/1911.03098/) (2019)
- [Ego2Hands: A Dataset for Egocentric Two-hand Segmentation and Detection](https://mlsanity.com/paper/2011.07252/) (2020)
- [Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labelling](https://mlsanity.com/paper/2003.12352/) (2020)
- [Using CNNs For Users Segmentation In Video See-Through Augmented Virtuality](https://mlsanity.com/paper/2001.00487/) (2020)

## Abstract (arXiv)

This paper presents a new framework for human body part segmentation based on Deep Convolutional Neural Networks trained using only synthetic data. The proposed approach achieves cutting-edge results without the need of training the models with real annotated data of human body parts. Our contributions include a data generation pipeline, that exploits a game engine for the creation of the synthetic data used for training the network, and a novel pre-processing module, that combines edge response maps and adaptive histogram equalization to guide the network to learn the shape of the human body parts ensuring robustness to changes in the illumination conditions. For selecting the best candidate architecture, we perform exhaustive tests on manually annotated images of real human body limbs. We further compare our method against several high-end commercial segmentation tools on the body parts segmentation task. The results show that our method outperforms the other models by a significant margin. Finally, we present an ablation study to validate our pre-processing module. With this paper, we release an implementation of the proposed approach along with the acquired datasets.