# Attention-based Part Assembly for 3D Volumetric Shape Modeling

arXiv: https://arxiv.org/abs/2304.10986 · HTML: https://mlsanity.com/paper/2304.10986/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 53 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Learning a Predictable and Generative Vector Representation for Objects](https://mlsanity.com/paper/1603.08637/) (2016)
- [Generative and Discriminative Voxel Modeling with Convolutional Neural Networks](https://mlsanity.com/paper/1608.04236/) (2016)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation](https://mlsanity.com/paper/1908.01491/) (2019)
- [BAE-NET: Branched Autoencoder for Shape Co-Segmentation](https://mlsanity.com/paper/1903.11228/) (2019)
- [3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks](https://mlsanity.com/paper/1708.01648/) (2017)
- [Leveraging 2D Data to Learn Textured 3D Mesh Generation](https://mlsanity.com/paper/2004.04180/) (2020)
- [StructureNet: Hierarchical Graph Networks for 3D Shape Generation](https://mlsanity.com/paper/1908.00575/) (2019)
- [Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction](https://mlsanity.com/paper/1801.03910/) (2018)
- [C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds](https://mlsanity.com/paper/1912.07009/) (2019)
- [D2IM-Net: Learning Detail Disentangled Implicit Fields from Single Images](https://mlsanity.com/paper/2012.06650/) (2020)
- [Learning Part Generation and Assembly for Structure-aware Shape Synthesis](https://mlsanity.com/paper/1906.06693/) (2019)
- [Composite Shape Modeling via Latent Space Factorization](https://mlsanity.com/paper/1901.02968/) (2019)
- [COALESCE: Component Assembly by Learning to Synthesize Connections](https://mlsanity.com/paper/2008.01936/) (2020)

## Built on by (most cited first, 3 of 3)

- [Point Cloud Part Editing: Segmentation, Generation, Assembly, and Selection](https://mlsanity.com/paper/2312.11867/) (2023)
- [Aligning Instance-Semantic Sparse Representation towards Unsupervised Object Segmentation and Shape Abstraction with Repeatable Primitives](https://mlsanity.com/paper/2503.06947/) (2025)
- [Exploiting Topological Priors for Boosting Point Cloud Generation](https://mlsanity.com/paper/2403.10962/) (2024)

## Abstract (arXiv)

Modeling a 3D volumetric shape as an assembly of decomposed shape parts is much more challenging, but semantically more valuable than direct reconstruction from a full shape representation. The neural network needs to implicitly learn part relations coherently, which is typically performed by dedicated network layers that can generate transformation matrices for each part. In this paper, we propose a VoxAttention network architecture for attention-based part assembly. We further propose a variant of using channel-wise part attention and show the advantages of this approach. Experimental results show that our method outperforms most state-of-the-art methods for the part relation-aware 3D shape modeling task.