# NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generation arXiv: https://arxiv.org/abs/2403.18241 ยท HTML: https://mlsanity.com/paper/2403.18241/ ## Key facts - Cited by 13 papers in the map (data through 14 Oct 2026). - Builds on 66 of its 79 references that are in the map. - First posted on arXiv 27 Mar 2024. ## Builds on (most cited first, 30 of 66) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016) - [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022) - [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015) - [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019) - [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020) - [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020) - [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023) - [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021) - [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018) - [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016) - [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023) - [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020) - [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021) - [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016) - [LION: Latent Point Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2210.06978/) (2022) - [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021) - [PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows](https://mlsanity.com/paper/1906.12320/) (2019) - [3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models](https://mlsanity.com/paper/2301.11445/) (2023) - [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022) - [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019) - [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019) - [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022) - [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](https://mlsanity.com/paper/2212.04493/) (2022) ## Built on by (most cited first, 13 of 13) - [Amodal3R: Amodal 3D Reconstruction from Occluded 2D Images](https://mlsanity.com/paper/2503.13439/) (2025) - [SceneMaker: Open-set 3D Scene Generation with Decoupled De-occlusion and Pose Estimation Model](https://mlsanity.com/paper/2512.10957/) (2025) - [DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data](https://mlsanity.com/paper/2406.04322/) (2024) - [Token Perturbation Guidance for Diffusion Models](https://mlsanity.com/paper/2506.10036/) (2025) - [LAM3D: Large Image-Point-Cloud Alignment Model for 3D Reconstruction from Single Image](https://mlsanity.com/paper/2405.15622/) (2024) - [NumGrad-Pull: Numerical Gradient Guided Tri-plane Representation for Surface Reconstruction from Point Clouds](https://mlsanity.com/paper/2411.17392/) (2024) - [DreamCharacter-1: From 3D Generative Foundation Models to Product-Ready Character Generation](https://mlsanity.com/paper/2607.07817/) (2026) - [Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image](https://mlsanity.com/paper/2511.11074/) (2025) - [VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification](https://mlsanity.com/paper/2406.05543/) (2024) - [3D Shape Completion with Test-Time Training](https://mlsanity.com/paper/2410.18668/) (2024) - [Squeeze3D: Extreme Neural Compression with Latent Space Bridging](https://mlsanity.com/paper/2506.07932/) (2025) - [Order Matters: 3D Shape Generation from Sequential VR Sketches](https://mlsanity.com/paper/2512.04761/) (2025) - [Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch](https://mlsanity.com/paper/2604.09100/) (2026) ## Abstract (arXiv) 3D shape generation aims to produce innovative 3D content adhering to specific conditions and constraints. Existing methods often decompose 3D shapes into a sequence of localized components, treating each element in isolation without considering spatial consistency. As a result, these approaches exhibit limited versatility in 3D data representation and shape generation, hindering their ability to generate highly diverse 3D shapes that comply with the specified constraints. In this paper, we introduce a novel spatial-aware 3D shape generation framework that leverages 2D plane representations for enhanced 3D shape modeling. To ensure spatial coherence and reduce memory usage, we incorporate a hybrid shape representation technique that directly learns a continuous signed distance field representation of the 3D shape using orthogonal 2D planes. Additionally, we meticulously enforce spatial correspondences across distinct planes using a transformer-based autoencoder structure, promoting the preservation of spatial relationships in the generated 3D shapes. This yields an algorithm that consistently outperforms state-of-the-art 3D shape generation methods on various tasks, including unconditional shape generation, multi-modal shape completion, single-view reconstruction, and text-to-shape synthesis. Our project page is available at https://weizheliu.github.io/NeuSDFusion/ .