# From CAD models to soft point cloud labels: An automatic annotation pipeline for cheaply supervised 3D semantic segmentation

arXiv: https://arxiv.org/abs/2302.03114 · HTML: https://mlsanity.com/paper/2302.03114/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 65 references that are in the map.
- First posted on arXiv 6 Feb 2023.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies](https://mlsanity.com/paper/2206.04670/) (2022)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review](https://mlsanity.com/paper/2005.09830/) (2020)
- [Making Better Mistakes: Leveraging Class Hierarchies with Deep Networks](https://mlsanity.com/paper/1912.09393/) (2019)
- [Deep Label Distribution Learning with Label Ambiguity](https://mlsanity.com/paper/1611.01731/) (2016)
- [Delving Deep into Label Smoothing](https://mlsanity.com/paper/2011.12562/) (2020)
- [Scribble-Supervised LiDAR Semantic Segmentation](https://mlsanity.com/paper/2203.08537/) (2022)
- [SalsaNet: Fast Road and Vehicle Segmentation in LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/1909.08291/) (2019)
- [Contrastive Boundary Learning for Point Cloud Segmentation](https://mlsanity.com/paper/2203.05272/) (2022)
- [Weakly Supervised Semantic Point Cloud Segmentation:Towards 10X Fewer Labels](https://mlsanity.com/paper/2004.04091/) (2020)
- [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021)
- [IntrA: 3D Intracranial Aneurysm Dataset for Deep Learning](https://mlsanity.com/paper/2003.02920/) (2020)
- [Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation](https://mlsanity.com/paper/1908.08854/) (2019)
- [SoftSeg: Advantages of soft versus binary training for image segmentation](https://mlsanity.com/paper/2011.09041/) (2020)
- [LATTE: Accelerating LiDAR Point Cloud Annotation via Sensor Fusion, One-Click Annotation, and Tracking](https://mlsanity.com/paper/1904.09085/) (2019)
- [Boosting LiDAR-based Semantic Labeling by Cross-Modal Training Data Generation](https://mlsanity.com/paper/1804.09915/) (2018)
- [Automatic labelling of urban point clouds using data fusion](https://mlsanity.com/paper/2108.13757/) (2021)
- [Navigation-Oriented Scene Understanding for Robotic Autonomy: Learning to Segment Driveability in Egocentric Images](https://mlsanity.com/paper/2109.07245/) (2021)
- [CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration](https://mlsanity.com/paper/2103.06911/) (2021)
- [Deep Workpiece Region Segmentation for Bin Picking](https://mlsanity.com/paper/1909.03462/) (2019)
- [Accurate Instance-Level CAD Model Retrieval in a Large-Scale Database](https://mlsanity.com/paper/2207.01339/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Robust Few-shot Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2309.11228/) (2023)

## Abstract (arXiv)

We propose a fully automatic annotation scheme that takes a raw 3D point cloud with a set of fitted CAD models as input and outputs convincing point-wise labels that can be used as cheap training data for point cloud segmentation. Compared with manual annotations, we show that our automatic labels are accurate while drastically reducing the annotation time and eliminating the need for manual intervention or dataset-specific parameters. Our labeling pipeline outputs semantic classes and soft point-wise object scores, which can either be binarized into standard one-hot-encoded labels, thresholded into weak labels with ambiguous points left unlabeled, or used directly as soft labels during training. We evaluate the label quality and segmentation performance of PointNet++ on a dataset of real industrial point clouds and Scan2CAD, a public dataset of indoor scenes. Our results indicate that reducing supervision in areas that are more difficult to label automatically is beneficial compared with the conventional approach of naively assigning a hard "best guess" label to every point.