# Organize Primitives into Semantic Parts: Reinforcement Reasoning for 3D Segmentation

arXiv: https://arxiv.org/abs/2610.04602 · HTML: https://mlsanity.com/paper/2610.04602/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 3 Oct 2026.

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts](https://mlsanity.com/paper/2211.05272/) (2022)
- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond](https://mlsanity.com/paper/2504.11451/) (2025)
- [Supervised Fitting of Geometric Primitives to 3D Point Clouds](https://mlsanity.com/paper/1811.08988/) (2018)
- [P3-SAM: Native 3D Part Segmentation](https://mlsanity.com/paper/2509.06784/) (2025)
- [Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives](https://mlsanity.com/paper/2307.05473/) (2023)
- [Find Any Part in 3D](https://mlsanity.com/paper/2411.13550/) (2024)
- [Learning to Group: A Bottom-Up Framework for 3D Part Discovery in Unseen Categories](https://mlsanity.com/paper/2002.06478/) (2020)
- [PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation](https://mlsanity.com/paper/2312.04016/) (2023)
- [3DCoMPaT++: An improved Large-scale 3D Vision Dataset for Compositional Recognition](https://mlsanity.com/paper/2310.18511/) (2023)
- [SuperDec: 3D Scene Decomposition with Superquadric Primitives](https://mlsanity.com/paper/2504.00992/) (2025)
- [RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures](https://mlsanity.com/paper/2201.12763/) (2022)
- [Marching-Primitives: Shape Abstraction from Signed Distance Function](https://mlsanity.com/paper/2303.13190/) (2023)
- [Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing](https://mlsanity.com/paper/2402.17464/) (2024)
- [PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data](https://mlsanity.com/paper/2509.21965/) (2025)
- [Primitive-based Shape Abstraction via Nonparametric Bayesian Inference](https://mlsanity.com/paper/2203.14714/) (2022)
- [PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer](https://mlsanity.com/paper/2505.04622/) (2025)
- [GeoSAM2: Unleashing the Power of SAM2 for 3D Part Segmentation](https://mlsanity.com/paper/2508.14036/) (2025)
- [DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Image](https://mlsanity.com/paper/2309.12594/) (2023)
- [Hierarchical clustering in particle physics through reinforcement learning](https://mlsanity.com/paper/2011.08191/) (2020)
- [ParSEL: Parameterized Shape Editing with Language](https://mlsanity.com/paper/2405.20319/) (2024)
- [Light-SQ: Structure-aware Shape Abstraction with Superquadrics for Generated Meshes](https://mlsanity.com/paper/2509.24986/) (2025)
- [PatchAlign3D: Local Feature Alignment for Dense 3D Shape Understanding](https://mlsanity.com/paper/2601.02457/) (2026)
- [GaussianGPT: Towards Autoregressive 3D Gaussian Scene Generation](https://mlsanity.com/paper/2603.26661/) (2026)

## Abstract (arXiv)

Primitive-based 3D segmentation offers a compact and explicit alternative to dense surface prediction, naturally supporting structural abstraction and boundary localization. However, geometric decomposition alone does not determine how primitives should be organized into semantic parts: a single part may span multiple primitives, while geometrically similar or touching primitives may belong to different parts. We therefore introduce RePart (Reinforcement Part Reasoning), which formulates primitive-to-part organization as a finite-horizon Markov decision process and learns semantic organization through trajectory-level reinforcement reasoning. RePart constructs a Composable Primitive Workspace from fine-grained superquadrics and applies a merge-and-stop policy whose decisions are optimized by their downstream effects on the resulting partition rather than local primitive compatibility. The inferred part identities are then mapped back to the original mesh through Boundary-Aware Surface Labeling, preserving accurate surface boundaries beyond the primitive approximation. On PartNet, RePart achieves the strongest results across all four aggregate partition metrics; on 3DCoMPaT++, it obtains the highest RI and SC without target-dataset fine-tuning. These results demonstrate that reinforcement reasoning provides an effective mechanism for organizing geometric primitives into semantic parts while retaining dense segmentation accuracy. Code is available at https://github.com/EngineeringAI-LAB/RePart.