# 3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds

arXiv: https://arxiv.org/abs/1707.06783 · HTML: https://mlsanity.com/paper/1707.06783/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2017.

## Built on by (most cited first, 12 of 12)

- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction](https://mlsanity.com/paper/2203.00494/) (2022)
- [Deep Learning-based 3D Point Cloud Classification: A Systematic Survey and Outlook](https://mlsanity.com/paper/2311.02608/) (2023)
- [Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariance](https://mlsanity.com/paper/2305.16314/) (2023)
- [UV-Based 3D Hand-Object Reconstruction with Grasp Optimization](https://mlsanity.com/paper/2211.13429/) (2022)
- [LRGNet: Learnable Region Growing for Class-Agnostic Point Cloud Segmentation](https://mlsanity.com/paper/2103.09160/) (2021)
- [Permutation-Invariant Set Autoencoders with Fixed-Size Embeddings for Multi-Agent Learning](https://mlsanity.com/paper/2302.12826/) (2023)
- [A Framework for Incentivized Collaborative Learning](https://mlsanity.com/paper/2305.17052/) (2023)
- [PRENet: A Plane-Fit Redundancy Encoding Point Cloud Sequence Network for Real-Time 3D Action Recognition](https://mlsanity.com/paper/2405.06929/) (2024)
- [Assisted Learning for Organizations with Limited Imbalanced Data](https://mlsanity.com/paper/2109.09307/) (2021)
- [On the Privacy Risks of Deploying Recurrent Neural Networks in Machine Learning Models](https://mlsanity.com/paper/2110.03054/) (2021)
- [Reinforcement Learning Based Multi-modal Feature Fusion Network for Novel Class Discovery](https://mlsanity.com/paper/2308.13801/) (2023)

## Abstract (arXiv)

Semantic parsing of large-scale 3D point clouds is an important research topic in computer vision and remote sensing fields. Most existing approaches utilize hand-crafted features for each modality independently and combine them in a heuristic manner. They often fail to consider the consistency and complementary information among features adequately, which makes them difficult to capture high-level semantic structures. The features learned by most of the current deep learning methods can obtain high-quality image classification results. However, these methods are hard to be applied to recognize 3D point clouds due to unorganized distribution and various point density of data. In this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural network (RNN) for an efficient semantic parsing of large-scale 3D point clouds. In our method, an eye window under control of the 3D CNN and DQN can localize and segment the points of the object class efficiently. The 3D CNN and Residual RNN further extract robust and discriminative features of the points in the eye window, and thus greatly enhance the parsing accuracy of large-scale point clouds. Our method provides an automatic process that maps the raw data to the classification results. It also integrates object localization, segmentation and classification into one framework. Experimental results demonstrate that the proposed method outperforms the state-of-the-art point cloud classification methods.