# Synthetic-to-real Composite Semantic Segmentation in Additive Manufacturing

arXiv: https://arxiv.org/abs/2210.07466 · HTML: https://mlsanity.com/paper/2210.07466/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 84 references that are in the map.
- First posted on arXiv 14 Oct 2022.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.13132/) (2022)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)
- [Synthetic Data for Deep Learning](https://mlsanity.com/paper/1909.11512/) (2019)
- [SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2204.08808/) (2022)
- [MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2103.05254/) (2021)
- [A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images](https://mlsanity.com/paper/1912.10230/) (2019)
- [Semantic-aware Grad-GAN for Virtual-to-Real Urban Scene Adaption](https://mlsanity.com/paper/1801.01726/) (2018)
- [Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings](https://mlsanity.com/paper/2011.12616/) (2020)
- [Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2105.10843/) (2021)
- [Deep Leaf Segmentation Using Synthetic Data](https://mlsanity.com/paper/1807.10931/) (2018)
- [Unsupervised Domain Adaptation for Semantic Image Segmentation: a Comprehensive Survey](https://mlsanity.com/paper/2112.03241/) (2021)
- [StRDAN: Synthetic-to-Real Domain Adaptation Network for Vehicle Re-Identification](https://mlsanity.com/paper/2004.12032/) (2020)
- [Synthetic-to-Real Domain Adaptation using Contrastive Unpaired Translation](https://mlsanity.com/paper/2203.09454/) (2022)
- [Toward Enabling a Reliable Quality Monitoring System for Additive Manufacturing Process using Deep Convolutional Neural Networks](https://mlsanity.com/paper/2003.08749/) (2020)
- [Automatic Volumetric Segmentation of Additive Manufacturing Defects with 3D U-Net](https://mlsanity.com/paper/2101.08993/) (2021)
- [Open Source Computer Vision-based Layer-wise 3D Printing Analysis](https://mlsanity.com/paper/2003.05660/) (2020)
- [Towards Smart Monitored AM: Open Source in-Situ Layer-wise 3D Printing Image Anomaly Detection Using Histograms of Oriented Gradients and a Physics-Based Rendering Engine](https://mlsanity.com/paper/2111.02703/) (2021)

## Built on by (most cited first, 1 of 1)

- [Open Stamped Parts Dataset](https://mlsanity.com/paper/2403.10369/) (2024)

## Abstract (arXiv)

The application of computer vision and machine learning methods in the field of additive manufacturing (AM) for semantic segmentation of the structural elements of 3-D printed products will improve real-time failure analysis systems and can potentially reduce the number of defects by enabling in situ corrections. This work demonstrates the possibilities of using physics-based rendering for labeled image dataset generation, as well as image-to-image translation capabilities to improve the accuracy of real image segmentation for AM systems. Multi-class semantic segmentation experiments were carried out based on the U-Net model and cycle generative adversarial network. The test results demonstrated the capacity of detecting such structural elements of 3-D printed parts as a top layer, infill, shell, and support. A basis for further segmentation system enhancement by utilizing image-to-image style transfer and domain adaptation technologies was also developed. The results indicate that using style transfer as a precursor to domain adaptation can significantly improve real 3-D printing image segmentation in situations where a model trained on synthetic data is the only tool available. The mean intersection over union (mIoU) scores for synthetic test datasets included 94.90% for the entire 3-D printed part, 73.33% for the top layer, 78.93% for the infill, 55.31% for the shell, and 69.45% for supports.