# 3DRIMR: 3D Reconstruction and Imaging via mmWave Radar based on Deep Learning

arXiv: https://arxiv.org/abs/2108.02858 · HTML: https://mlsanity.com/paper/2108.02858/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 17 of 17)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis](https://mlsanity.com/paper/1612.00101/) (2016)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [Improved Adversarial Systems for 3D Object Generation and Reconstruction](https://mlsanity.com/paper/1707.09557/) (2017)
- [VConv-DAE: Deep Volumetric Shape Learning Without Object Labels](https://mlsanity.com/paper/1604.03755/) (2016)
- [See Through Smoke: Robust Indoor Mapping with Low-cost mmWave Radar](https://mlsanity.com/paper/1911.00398/) (2019)
- [3D Object Reconstruction from a Single Depth View with Adversarial Learning](https://mlsanity.com/paper/1708.07969/) (2017)
- [RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials](https://mlsanity.com/paper/1901.01535/) (2019)
- [FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation](https://mlsanity.com/paper/1712.07262/) (2017)

## Built on by (most cited first, 16 of 16)

- [Diffusion Model is a Good Pose Estimator from 3D RF-Vision](https://mlsanity.com/paper/2403.16198/) (2024)
- [Enabling Visual Recognition at Radio Frequency](https://mlsanity.com/paper/2405.19516/) (2024)
- [R2P: A Deep Learning Model from mmWave Radar to Point Cloud](https://mlsanity.com/paper/2207.10690/) (2022)
- [DeepPoint: A Deep Learning Model for 3D Reconstruction in Point Clouds via mmWave Radar](https://mlsanity.com/paper/2109.09188/) (2021)
- [MITO: A Millimeter-Wave Dataset and Simulator for Non-Line-of-Sight Perception](https://mlsanity.com/paper/2502.10259/) (2025)
- [SHOW3D: Capturing Scenes of 3D Hands and Objects in the Wild](https://mlsanity.com/paper/2603.28760/) (2026)
- [3D Reconstruction of Multiple Objects by mmWave Radar on UAV](https://mlsanity.com/paper/2211.02150/) (2022)
- [mmPred: Radar-based Human Motion Prediction in the Dark](https://mlsanity.com/paper/2512.00345/) (2025)
- [Scalable RF Simulation in Generative 4D Worlds](https://mlsanity.com/paper/2508.12176/) (2025)
- [4DR P2T: 4D Radar Tensor Synthesis with Point Clouds](https://mlsanity.com/paper/2502.05550/) (2025)
- [Ego-Exo 3D Hand Tracking in the Wild with a Mobile Multi-Camera Rig](https://mlsanity.com/paper/2510.02601/) (2025)
- [mmHRI: Towards Privacy-Preserving Human-Robot Interaction with Millimeter-Wave Radar](https://mlsanity.com/paper/2609.34220/) (2026)
- [Deep Learning-based Cross-modal Reconstruction of Vehicle Target from Sparse 3D SAR Image](https://mlsanity.com/paper/2406.04158/) (2024)
- [Wave-Former: Through-Occlusion 3D Reconstruction via Wireless Shape Completion](https://mlsanity.com/paper/2511.14152/) (2025)
- [GeRaF: Neural Geometry Reconstruction from Radio Frequency Signals](https://mlsanity.com/paper/2605.29097/) (2026)
- [Seeing through boxes: Non-Line-of-Sight 3D Reconstruction from Radar Signals](https://mlsanity.com/paper/2605.29098/) (2026)

## Abstract (arXiv)

mmWave radar has been shown as an effective sensing technique in low visibility, smoke, dusty, and dense fog environment. However tapping the potential of radar sensing to reconstruct 3D object shapes remains a great challenge, due to the characteristics of radar data such as sparsity, low resolution, specularity, high noise, and multi-path induced shadow reflections and artifacts. In this paper we propose 3D Reconstruction and Imaging via mmWave Radar (3DRIMR), a deep learning based architecture that reconstructs 3D shape of an object in dense detailed point cloud format, based on sparse raw mmWave radar intensity data. The architecture consists of two back-to-back conditional GAN deep neural networks: the first generator network generates 2D depth images based on raw radar intensity data, and the second generator network outputs 3D point clouds based on the results of the first generator. The architecture exploits both convolutional neural network's convolutional operation (that extracts local structure neighborhood information) and the efficiency and detailed geometry capture capability of point clouds (other than costly voxelization of 3D space or distance fields). Our experiments have demonstrated 3DRIMR's effectiveness in reconstructing 3D objects, and its performance improvement over standard techniques.