# SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image

arXiv: https://arxiv.org/abs/2305.12646 · HTML: https://mlsanity.com/paper/2305.12646/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 61 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 30 of 30)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [PU-Net: Point Cloud Upsampling Network](https://mlsanity.com/paper/1801.06761/) (2018)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [PU-GAN: a Point Cloud Upsampling Adversarial Network](https://mlsanity.com/paper/1907.10844/) (2019)
- [Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images](https://mlsanity.com/paper/1511.02300/) (2015)
- [Patch-based Progressive 3D Point Set Upsampling](https://mlsanity.com/paper/1811.11286/) (2018)
- [PU-GCN: Point Cloud Upsampling using Graph Convolutional Networks](https://mlsanity.com/paper/1912.03264/) (2019)
- [Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis](https://mlsanity.com/paper/2008.03483/) (2020)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Fine Perceptive GANs for Brain MR Image Super-Resolution in Wavelet Domain](https://mlsanity.com/paper/2011.04145/) (2020)
- [Tensorizing GAN with High-Order Pooling for Alzheimer's Disease Assessment](https://mlsanity.com/paper/2008.00748/) (2020)
- [Improving Semantic Analysis on Point Clouds via Auxiliary Supervision of Local Geometric Priors](https://mlsanity.com/paper/2001.04803/) (2020)
- [Parts4Feature: Learning 3D Global Features from Generally Semantic Parts in Multiple Views](https://mlsanity.com/paper/1905.07506/) (2019)
- [3D Brain Reconstruction by Hierarchical Shape-Perception Network from a Single Incomplete Image](https://mlsanity.com/paper/2107.11010/) (2021)
- [One-stage Shape Instantiation from a Single 2D Image to 3D Point Cloud](https://mlsanity.com/paper/1907.10763/) (2019)
- [A Point Cloud Generative Model via Tree-Structured Graph Convolutions for 3D Brain Shape Reconstruction](https://mlsanity.com/paper/2107.09923/) (2021)
- [SCH-GAN: Semi-supervised Cross-modal Hashing by Generative Adversarial Network](https://mlsanity.com/paper/1802.02488/) (2018)
- [Probabilistic Point Cloud Reconstructions for Vertebral Shape Analysis](https://mlsanity.com/paper/1907.09254/) (2019)
- [Instantiation-Net: 3D Mesh Reconstruction from Single 2D Image for Right Ventricle](https://mlsanity.com/paper/1909.08986/) (2019)

## Abstract (arXiv)

In minimally-invasive brain surgeries with indirect and narrow operating environments, 3D brain reconstruction is crucial. However, as requirements of accuracy for some new minimally-invasive surgeries (such as brain-computer interface surgery) are higher and higher, the outputs of conventional 3D reconstruction, such as point cloud (PC), are facing the challenges that sample points are too sparse and the precision is insufficient. On the other hand, there is a scarcity of high-density point cloud datasets, which makes it challenging to train models for direct reconstruction of high-density brain point clouds. In this work, a novel model named stereoscopic-aware graph generative adversarial network (SG-GAN) with two stages is proposed to generate fine high-density PC conditioned on a single image. The Stage-I GAN sketches the primitive shape and basic structure of the organ based on the given image, yielding Stage-I point clouds. The Stage-II GAN takes the results from Stage-I and generates high-density point clouds with detailed features. The Stage-II GAN is capable of correcting defects and restoring the detailed features of the region of interest (ROI) through the up-sampling process. Furthermore, a parameter-free-attention-based free-transforming module is developed to learn the efficient features of input, while upholding a promising performance. Comparing with the existing methods, the SG-GAN model shows superior performance in terms of visual quality, objective measurements, and performance in classification, as demonstrated by comprehensive results measured by several evaluation metrics including PC-to-PC error and Chamfer distance.