# Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion

arXiv: https://arxiv.org/abs/2506.17074 · HTML: https://mlsanity.com/paper/2506.17074/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 58 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 30 of 31)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [Structured 3D Latents for Scalable and Versatile 3D Generation](https://mlsanity.com/paper/2412.01506/) (2024)
- [Point-E: A System for Generating 3D Point Clouds from Complex Prompts](https://mlsanity.com/paper/2212.08751/) (2022)
- [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023)
- [CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets](https://mlsanity.com/paper/2406.13897/) (2024)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models](https://mlsanity.com/paper/2502.06608/) (2025)
- [Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer](https://mlsanity.com/paper/2405.14832/) (2024)
- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies](https://mlsanity.com/paper/2312.03806/) (2023)
- [CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner](https://mlsanity.com/paper/2405.14979/) (2024)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [Using Shape to Categorize: Low-Shot Learning with an Explicit Shape Bias](https://mlsanity.com/paper/2101.07296/) (2021)
- [CAST: Component-Aligned 3D Scene Reconstruction from an RGB Image](https://mlsanity.com/paper/2502.12894/) (2025)
- [SAMPart3D: Segment Any Part in 3D Objects](https://mlsanity.com/paper/2411.07184/) (2024)
- [Generative 3D Part Assembly via Dynamic Graph Learning](https://mlsanity.com/paper/2006.07793/) (2020)
- [PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models](https://mlsanity.com/paper/2312.11417/) (2023)
- [PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Models](https://mlsanity.com/paper/2412.18608/) (2024)
- [HoloPart: Generative 3D Part Amodal Segmentation](https://mlsanity.com/paper/2504.07943/) (2025)
- [Part123: Part-aware 3D Reconstruction from a Single-view Image](https://mlsanity.com/paper/2405.16888/) (2024)
- [OctFusion: Octree-based Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2408.14732/) (2024)
- [RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly](https://mlsanity.com/paper/2107.12859/) (2021)
- [Jigsaw: Learning to Assemble Multiple Fractured Objects](https://mlsanity.com/paper/2305.17975/) (2023)
- [Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing](https://mlsanity.com/paper/2402.17464/) (2024)
- [Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers](https://mlsanity.com/paper/2312.12340/) (2023)

## Built on by (most cited first, 12 of 12)

- [VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space](https://mlsanity.com/paper/2508.19247/) (2025)
- [SegviGen: Repurposing 3D Generative Model for Part Segmentation](https://mlsanity.com/paper/2603.16869/) (2026)
- [Part-X-MLLM: Part-aware 3D Multimodal Large Language Model](https://mlsanity.com/paper/2511.13647/) (2025)
- [Repurposing 3D Generative Model for Autoregressive Layout Generation](https://mlsanity.com/paper/2604.16299/) (2026)
- [Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data](https://mlsanity.com/paper/2604.13688/) (2026)
- [SARe: Structure-Aware Generative 3D Fragment Reassembly](https://mlsanity.com/paper/2603.21611/) (2026)
- [EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning](https://mlsanity.com/paper/2607.07187/) (2026)
- [STEP-Parts: Geometric Partitioning of Boundary Representations for Large-Scale CAD Processing](https://mlsanity.com/paper/2604.14927/) (2026)
- [Curvature-Aware Captioning:Leveraging Geodesic Attention for 3D Scene Understanding](https://mlsanity.com/paper/2605.08808/) (2026)
- [Arbor: Explicit Geometric Conditioning for Controllable 3D Asset Generation](https://mlsanity.com/paper/2606.23514/) (2026)
- [AssemblyWorld: Rethinking 3D Assembly with General-Purpose Agents](https://mlsanity.com/paper/2609.40353/) (2026)
- [LLM2R: From Large Language Models to 3D Reassemblers](https://mlsanity.com/paper/2610.18604/) (2026)

## Abstract (arXiv)

We present Assembler, a scalable and generalizable framework for 3D part assembly that reconstructs complete objects from input part meshes and a reference image. Unlike prior approaches that mostly rely on deterministic part pose prediction and category-specific training, Assembler is designed to handle diverse, in-the-wild objects with varying part counts, geometries, and structures. It addresses the core challenges of scaling to general 3D part assembly through innovations in task formulation, representation, and data. First, Assembler casts part assembly as a generative problem and employs diffusion models to sample plausible configurations, effectively capturing ambiguities arising from symmetry, repeated parts, and multiple valid assemblies. Second, we introduce a novel shape-centric representation based on sparse anchor point clouds, enabling scalable generation in Euclidean space rather than SE(3) pose prediction. Third, we construct a large-scale dataset of over 320K diverse part-object assemblies using a synthesis and filtering pipeline built on existing 3D shape repositories. Assembler achieves state-of-the-art performance on PartNet and is the first to demonstrate high-quality assembly for complex, real-world objects. Based on Assembler, we further introduce an interesting part-aware 3D modeling system that generates high-resolution, editable objects from images, demonstrating potential for interactive and compositional design. Project page: https://assembler3d.github.io