# DeepPoint: A Deep Learning Model for 3D Reconstruction in Point Clouds via mmWave Radar

arXiv: https://arxiv.org/abs/2109.09188 · HTML: https://mlsanity.com/paper/2109.09188/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 27 references that are in the map.
- First posted on arXiv 19 Sep 2021.

## Builds on (most cited first, 16 of 16)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis](https://mlsanity.com/paper/1612.00101/) (2016)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [Improved Adversarial Systems for 3D Object Generation and Reconstruction](https://mlsanity.com/paper/1707.09557/) (2017)
- [VConv-DAE: Deep Volumetric Shape Learning Without Object Labels](https://mlsanity.com/paper/1604.03755/) (2016)
- [See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar](https://mlsanity.com/paper/1911.00398/) (2019)
- [3D Object Reconstruction from a Single Depth View with Adversarial Learning](https://mlsanity.com/paper/1708.07969/) (2017)
- [RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials](https://mlsanity.com/paper/1901.01535/) (2019)
- [3DRIMR: 3D Reconstruction and Imaging via mmWave Radar based on Deep Learning](https://mlsanity.com/paper/2108.02858/) (2021)
- [LIDAUS: Localization of IoT Device via Anchor UAV SLAM](https://mlsanity.com/paper/2108.02718/) (2021)

## Built on by (most cited first, 5 of 5)

- [DART: Implicit Doppler Tomography for Radar Novel View Synthesis](https://mlsanity.com/paper/2403.03896/) (2024)
- [Enabling Visual Recognition at Radio Frequency](https://mlsanity.com/paper/2405.19516/) (2024)
- [Spatio-Temporal 3D Point Clouds from WiFi-CSI Data via Transformer Networks](https://mlsanity.com/paper/2410.16303/) (2024)
- [Scalable RF Simulation in Generative 4D Worlds](https://mlsanity.com/paper/2508.12176/) (2025)
- [Generative Adversarial Synthesis of Radar Point Cloud Scenes](https://mlsanity.com/paper/2410.13526/) (2024)

## Abstract (arXiv)

Recent research has shown that mmWave radar sensing is effective for object detection in low visibility environments, which makes it an ideal technique in autonomous navigation systems such as autonomous vehicles. However, due to the characteristics of radar signals such as sparsity, low resolution, specularity, and high noise, it is still quite challenging to reconstruct 3D object shapes via mmWave radar sensing. Built on our recent proposed 3DRIMR (3D Reconstruction and Imaging via mmWave Radar), we introduce in this paper DeepPoint, a deep learning model that generates 3D objects in point cloud format that significantly outperforms the original 3DRIMR design. The model adopts a conditional Generative Adversarial Network (GAN) based deep neural network architecture. It takes as input the 2D depth images of an object generated by 3DRIMR's Stage 1, and outputs smooth and dense 3D point clouds of the object. The model consists of a novel generator network that utilizes a sequence of DeepPoint blocks or layers to extract essential features of the union of multiple rough and sparse input point clouds of an object when observed from various viewpoints, given that those input point clouds may contain many incorrect points due to the imperfect generation process of 3DRIMR's Stage 1. The design of DeepPoint adopts a deep structure to capture the global features of input point clouds, and it relies on an optimally chosen number of DeepPoint blocks and skip connections to achieve performance improvement over the original 3DRIMR design. Our experiments have demonstrated that this model significantly outperforms the original 3DRIMR and other standard techniques in reconstructing 3D objects.