# StructRe: Rewriting for Structured Shape Modeling

arXiv: https://arxiv.org/abs/2311.17510 · HTML: https://mlsanity.com/paper/2311.17510/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 67 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [In-Context Learning Creates Task Vectors](https://mlsanity.com/paper/2310.15916/) (2023)
- [On the Binding Problem in Artificial Neural Networks](https://mlsanity.com/paper/2012.05208/) (2020)
- [AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation](https://mlsanity.com/paper/2203.09516/) (2022)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Models](https://mlsanity.com/paper/2212.01558/) (2022)
- [SDM-NET: Deep Generative Network for Structured Deformable Mesh](https://mlsanity.com/paper/1908.04520/) (2019)
- [DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape Generation](https://mlsanity.com/paper/2008.05440/) (2020)
- [StructEdit: Learning Structural Shape Variations](https://mlsanity.com/paper/1911.11098/) (2019)
- [ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives](https://mlsanity.com/paper/2305.05661/) (2023)
- [SAGNet:Structure-aware Generative Network for 3D-Shape Modeling](https://mlsanity.com/paper/1808.03981/) (2018)
- [PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions](https://mlsanity.com/paper/2003.08624/) (2020)
- [ANISE: Assembly-based Neural Implicit Surface rEconstruction](https://mlsanity.com/paper/2205.13682/) (2022)
- [LSD-StructureNet: Modeling Levels of Structural Detail in 3D Part Hierarchies](https://mlsanity.com/paper/2108.13459/) (2021)
- [The Shape Part Slot Machine: Contact-based Reasoning for Generating 3D Shapes from Parts](https://mlsanity.com/paper/2112.00584/) (2021)
- [Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing](https://mlsanity.com/paper/2211.00382/) (2022)
- [An A*-algorithm for the Unordered Tree Edit Distance with Custom Costs](https://mlsanity.com/paper/2108.00953/) (2021)

## Built on by (most cited first, 1 of 1)

- [ComboStoc: Combinatorial Stochasticity for Diffusion Generative Models](https://mlsanity.com/paper/2405.13729/) (2024)

## Abstract (arXiv)

Man-made 3D shapes are naturally organized in parts and hierarchies; such structures provide important constraints for shape reconstruction and generation. Modeling shape structures is difficult, because there can be multiple hierarchies for a given shape, causing ambiguity, and across different categories the shape structures are correlated with semantics, limiting generalization. We present StructRe, a structure rewriting system, as a novel approach to structured shape modeling. Given a 3D object represented by points and components, StructRe can rewrite it upward into more concise structures, or downward into more detailed structures; by iterating the rewriting process, hierarchies are obtained. Such a localized rewriting process enables probabilistic modeling of ambiguous structures and robust generalization across object categories. We train StructRe on PartNet data and show its generalization to cross-category and multiple object hierarchies, and test its extension to ShapeNet. We also demonstrate the benefits of probabilistic and generalizable structure modeling for shape reconstruction, generation and editing tasks.