# 3D Brain Reconstruction by Hierarchical Shape-Perception Network from a Single Incomplete Image arXiv: https://arxiv.org/abs/2107.11010 ยท HTML: https://mlsanity.com/paper/2107.11010/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 73 references that are in the map. - First posted on arXiv 23 Jul 2021. ## Builds on (most cited first, 24 of 24) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017) - [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018) - [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017) - [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016) - [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018) - [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016) - [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018) - [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018) - [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019) - [Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images](https://mlsanity.com/paper/1511.02300/) (2015) - [Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis](https://mlsanity.com/paper/2008.03483/) (2020) - [Fine Perceptive GANs for Brain MR Image Super-Resolution in Wavelet Domain](https://mlsanity.com/paper/2011.04145/) (2020) - [Tensorizing GAN with High-Order Pooling for Alzheimer's Disease Assessment](https://mlsanity.com/paper/2008.00748/) (2020) - [Characterization Multimodal Connectivity of Brain Network by Hypergraph GAN for Alzheimer's Disease Analysis](https://mlsanity.com/paper/2107.09953/) (2021) - [Improving Semantic Analysis on Point Clouds via Auxiliary Supervision of Local Geometric Priors](https://mlsanity.com/paper/2001.04803/) (2020) - [Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views](https://mlsanity.com/paper/1905.07506/) (2019) - [One-stage Shape Instantiation from a Single 2D Image to 3D Point Cloud](https://mlsanity.com/paper/1907.10763/) (2019) - [A Point Cloud Generative Model via Tree-Structured Graph Convolutions for 3D Brain Shape Reconstruction](https://mlsanity.com/paper/2107.09923/) (2021) - [SCH-GAN: Semi-supervised Cross-modal Hashing by Generative Adversarial Network](https://mlsanity.com/paper/1802.02488/) (2018) - [Probabilistic Point Cloud Reconstructions for Vertebral Shape Analysis](https://mlsanity.com/paper/1907.09254/) (2019) ## Built on by (most cited first, 10 of 10) - [MedPrompt: Cross-Modal Prompting for Multi-Task Medical Image Translation](https://mlsanity.com/paper/2310.02663/) (2023) - [CATD: Unified Representation Learning for EEG-to-fMRI Cross-Modal Generation](https://mlsanity.com/paper/2408.00777/) (2024) - [Inflating 2D Convolution Weights for Efficient Generation of 3D Medical Images](https://mlsanity.com/paper/2208.03934/) (2022) - [Brain Structure-Function Fusing Representation Learning using Adversarial Decomposed-VAE for Analyzing MCI](https://mlsanity.com/paper/2305.14404/) (2023) - [Multi-modal Mood Reader: Pre-trained Model Empowers Cross-Subject Emotion Recognition](https://mlsanity.com/paper/2405.19373/) (2024) - [From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction](https://mlsanity.com/paper/2505.03599/) (2025) - [BDHT: Generative AI Enables Causality Analysis for Mild Cognitive Impairment](https://mlsanity.com/paper/2312.09022/) (2023) - [ConnectomeDiffuser: Generative AI Enables Brain Network Construction from Diffusion Tensor Imaging](https://mlsanity.com/paper/2505.22683/) (2025) - [SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image](https://mlsanity.com/paper/2305.12646/) (2023) - [High-Fidelity 3D Geometric Reconstruction of Pelvic Organs from MRI: A Hybrid Deep Learning and Iterative Optimization Approach](https://mlsanity.com/paper/2606.17836/) (2026) ## Abstract (arXiv) 3D shape reconstruction is essential in the navigation of minimally-invasive and auto robot-guided surgeries whose operating environments are indirect and narrow, and there have been some works that focused on reconstructing the 3D shape of the surgical organ through limited 2D information available. However, the lack and incompleteness of such information caused by intraoperative emergencies (such as bleeding) and risk control conditions have not been considered. In this paper, a novel hierarchical shape-perception network (HSPN) is proposed to reconstruct the 3D point clouds (PCs) of specific brains from one single incomplete image with low latency. A branching predictor and several hierarchical attention pipelines are constructed to generate point clouds that accurately describe the incomplete images and then complete these point clouds with high quality. Meanwhile, attention gate blocks (AGBs) are designed to efficiently aggregate geometric local features of incomplete PCs transmitted by hierarchical attention pipelines and internal features of reconstructing point clouds. With the proposed HSPN, 3D shape perception and completion can be achieved spontaneously. Comprehensive results measured by Chamfer distance and PC-to-PC error demonstrate that the performance of the proposed HSPN outperforms other competitive methods in terms of qualitative displays, quantitative experiment, and classification evaluation.