# Region-Transformer: Self-Attention Region Based Class-Agnostic Point Cloud Segmentation

arXiv: https://arxiv.org/abs/2403.01407 · HTML: https://mlsanity.com/paper/2403.01407/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 35 references that are in the map.
- First posted on arXiv 3 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds](https://mlsanity.com/paper/1906.01140/) (2019)
- [Unseen Object Instance Segmentation for Robotic Environments](https://mlsanity.com/paper/2007.08073/) (2020)
- [Few-shot 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2006.12052/) (2020)
- [3D Point Cloud Processing and Learning for Autonomous Driving](https://mlsanity.com/paper/2003.00601/) (2020)
- [JSIS3D: Joint Semantic-Instance Segmentation of 3D Point Clouds with Multi-Task Pointwise Networks and Multi-Value Conditional Random Fields](https://mlsanity.com/paper/1904.00699/) (2019)
- [Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2202.13589/) (2022)
- [Know What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds](https://mlsanity.com/paper/1810.01151/) (2018)
- [Learning to Optimally Segment Point Clouds](https://mlsanity.com/paper/1912.04976/) (2019)
- [Fast and Accurate Normal Estimation for Point Cloud via Patch Stitching](https://mlsanity.com/paper/2103.16066/) (2021)
- [Class-Agnostic Segmentation Loss and Its Application to Salient Object Detection and Segmentation](https://mlsanity.com/paper/2010.14793/) (2020)
- [Curvature of point clouds through principal component analysis](https://mlsanity.com/paper/2106.09972/) (2021)
- [Projection-based Point Convolution for Efficient Point Cloud Segmentation](https://mlsanity.com/paper/2202.01991/) (2022)

## Abstract (arXiv)

Point cloud segmentation, which helps us understand the environment of specific structures and objects, can be performed in class-specific and class-agnostic ways. We propose a novel region-based transformer model called Region-Transformer for performing class-agnostic point cloud segmentation. The model utilizes a region-growth approach and self-attention mechanism to iteratively expand or contract a region by adding or removing points. It is trained on simulated point clouds with instance labels only, avoiding semantic labels. Attention-based networks have succeeded in many previous methods of performing point cloud segmentation. However, a region-growth approach with attention-based networks has yet to be used to explore its performance gain. To our knowledge, we are the first to use a self-attention mechanism in a region-growth approach. With the introduction of self-attention to region-growth that can utilize local contextual information of neighborhood points, our experiments demonstrate that the Region-Transformer model outperforms previous class-agnostic and class-specific methods on indoor datasets regarding clustering metrics. The model generalizes well to large-scale scenes. Key advantages include capturing long-range dependencies through self-attention, avoiding the need for semantic labels during training, and applicability to a variable number of objects. The Region-Transformer model represents a promising approach for flexible point cloud segmentation with applications in robotics, digital twinning, and autonomous vehicles.