# From Part to Whole: 3D Generative World Model with an Adaptive Structural Hierarchy arXiv: https://arxiv.org/abs/2603.21557 ยท HTML: https://mlsanity.com/paper/2603.21557/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Video-generation world models and simulators, one of 11 directions in World models. - Builds on 25 of its 28 references that are in the map. - First posted on arXiv 23 Mar 2026. ## Placed in - World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/) ## Builds on (most cited first, 25 of 25) - [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023) - [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015) - [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020) - [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023) - [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023) - [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020) - [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021) - [TripoSR: Fast 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/2403.02151/) (2024) - [TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models](https://mlsanity.com/paper/2502.06608/) (2025) - [ATISS: Autoregressive Transformers for Indoor Scene Synthesis](https://mlsanity.com/paper/2110.03675/) (2021) - [MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation](https://mlsanity.com/paper/2412.03558/) (2024) - [Context-aware Biaffine Localizing Network for Temporal Sentence Grounding](https://mlsanity.com/paper/2103.11555/) (2021) - [PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers](https://mlsanity.com/paper/2506.05573/) (2025) - [HoloPart: Generative 3D Part Amodal Segmentation](https://mlsanity.com/paper/2504.07943/) (2025) - [Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning](https://mlsanity.com/paper/2107.01886/) (2021) - [Efficient Part-level 3D Object Generation via Dual Volume Packing](https://mlsanity.com/paper/2506.09980/) (2025) - [X-Part: high fidelity and structure coherent shape decomposition](https://mlsanity.com/paper/2509.08643/) (2025) - [Imperceptible Transfer Attack and Defense on 3D Point Cloud Classification](https://mlsanity.com/paper/2111.10990/) (2021) - [A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directions](https://mlsanity.com/paper/2406.05785/) (2024) - [Deep Point Set Resampling via Gradient Fields](https://mlsanity.com/paper/2111.02045/) (2021) - [Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing](https://mlsanity.com/paper/2402.17464/) (2024) - [WorldGrow: Generating Infinite 3D World](https://mlsanity.com/paper/2510.21682/) (2025) - [Multi-scale Latent Point Consistency Models for 3D Shape Generation](https://mlsanity.com/paper/2412.19413/) (2024) - [Geometry and Perception Guided Gaussians for Multiview-consistent 3D Generation from a Single Image](https://mlsanity.com/paper/2506.21152/) (2025) - [Behave Your Motion: Habit-preserved Cross-category Animal Motion Transfer](https://mlsanity.com/paper/2507.07394/) (2025) ## Built on by (most cited first, 1 of 1) - [View-Consistent 3D Scene Editing via Dual-Path Structural Correspondense and Semantic Continuity](https://mlsanity.com/paper/2604.17801/) (2026) ## Abstract (arXiv) Single-image 3D generation lies at the core of vision-to-graphics models in the real world. However, it remains a fundamental challenge to achieve reliable generalization across diverse semantic categories and highly variable structural complexity under sparse supervision. Existing approaches typically model objects in a monolithic manner or rely on a fixed number of parts, including recent part-aware models such as PartCrafter, which still require a labor-intensive user-specified part count. Such designs easily lead to overfitting, fragmented or missing structural components, and limited compositional generalization when encountering novel object layouts. To this end, this paper rethinks single-image 3D generation as learning an adaptive part-whole hierarchy in the flexible 3D latent space. We present a novel part-to-whole 3D generative world model that autonomously discovers latent structural slots by inferring soft and compositional masks directly from image tokens. Specifically, an adaptive slot-gating mechanism dynamically determines the slot-wise activation probabilities and smoothly consolidates redundant slots within different objects, ensuring that the emergent structure remains compact yet expressive across categories. Each distilled slot is then aligned to a learnable, class-agnostic prototype bank, enabling powerful cross-category shape sharing and denoising through universal geometric prototypes in the real world. Furthermore, a lightweight 3D denoiser is introduced to reconstruct geometry and appearance via unified diffusion objectives. Experiments show consistent gains in cross-category transfer and part-count extrapolation, and ablations confirm complementary benefits of the prototype bank for shape-prior sharing as well as slot-gating for structural adaptation.