# Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly

arXiv: https://arxiv.org/abs/2301.05033 · HTML: https://mlsanity.com/paper/2301.05033/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 59 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 30 of 34)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs](https://mlsanity.com/paper/1704.02901/) (2017)
- [Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Data](https://mlsanity.com/paper/1909.00889/) (2019)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [Tangent Convolutions for Dense Prediction in 3D](https://mlsanity.com/paper/1807.02443/) (2018)
- [Deep Parametric Continuous Convolutional Neural Networks](https://mlsanity.com/paper/2101.06742/) (2021)
- [Pointwise Convolutional Neural Networks](https://mlsanity.com/paper/1712.05245/) (2017)
- [Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation](https://mlsanity.com/paper/1804.06534/) (2018)
- [Deep Projective 3D Semantic Segmentation](https://mlsanity.com/paper/1705.03428/) (2017)
- [BlenderProc](https://mlsanity.com/paper/1911.01911/) (2019)
- [PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/1807.00652/) (2018)
- [Meta Reinforcement Learning for Sim-to-real Domain Adaptation](https://mlsanity.com/paper/1909.12906/) (2019)
- [Auto-Tuned Sim-to-Real Transfer](https://mlsanity.com/paper/2104.07662/) (2021)
- [EDEN: Multimodal Synthetic Dataset of Enclosed GarDEN Scenes](https://mlsanity.com/paper/2011.04389/) (2020)

## Built on by (most cited first, 16 of 16)

- [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024)
- [Neural MP: A Generalist Neural Motion Planner](https://mlsanity.com/paper/2409.05864/) (2024)
- [Advancing 3D Point Cloud Understanding through Deep Transfer Learning: A Comprehensive Survey](https://mlsanity.com/paper/2407.17877/) (2024)
- [Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey](https://mlsanity.com/paper/2402.12923/) (2024)
- [Rethinking Attention Module Design for Point Cloud Analysis](https://mlsanity.com/paper/2407.19294/) (2024)
- [JiSAM: Alleviate Labeling Burden and Corner Case Problems in Autonomous Driving via Minimal Real-World Data](https://mlsanity.com/paper/2503.08422/) (2025)
- [TopoCut: Learning Multi-Step Cutting with Spectral Rewards and Discrete Diffusion Policies](https://mlsanity.com/paper/2509.19712/) (2025)
- [TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding](https://mlsanity.com/paper/2511.07007/) (2025)
- [SAMBLE: Shape-Specific Point Cloud Sampling for an Optimal Trade-Off Between Local Detail and Global Uniformity](https://mlsanity.com/paper/2504.19581/) (2025)
- [Mind the Domain Gap: Measuring the Domain Gap Between Real-World and Synthetic Point Clouds for Automated Driving Development](https://mlsanity.com/paper/2505.17959/) (2025)
- [Resolving Primitive-Sharing Ambiguity in Long-Tailed TLS-Based Industrial MEP Point Cloud Segmentation via Spatial Context Constraints](https://mlsanity.com/paper/2601.19128/) (2026)
- [3D-TexSeg: Unsupervised Segmentation of 3D Texture using Mutual Transformer Learning](https://mlsanity.com/paper/2311.10651/) (2023)
- [SuDA: Support-based Domain Adaptation for Sim2Real Motion Capture with Flexible Sensors](https://mlsanity.com/paper/2405.16152/) (2024)
- [6D Pose Estimation on Point Cloud Data through Prior Knowledge Integration: A Case Study in Autonomous Disassembly](https://mlsanity.com/paper/2505.24669/) (2025)
- [Enhancing Human-Robot Collaboration: A Sim2Real Domain Adaptation Algorithm for Point Cloud Segmentation in Industrial Environments](https://mlsanity.com/paper/2506.09552/) (2025)
- [Open-vocabulary 3D scene perception in industrial environments](https://mlsanity.com/paper/2602.19823/) (2026)

## Abstract (arXiv)

On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.