# Planning with Learned Dynamic Model for Unsupervised Point Cloud Registration

arXiv: https://arxiv.org/abs/2108.02613 · HTML: https://mlsanity.com/paper/2108.02613/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 18 of 18)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Learning Latent Dynamics for Planning from Pixels](https://mlsanity.com/paper/1811.04551/) (2018)
- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [RPM-Net: Robust Point Matching using Learned Features](https://mlsanity.com/paper/2003.13479/) (2020)
- [The Perfect Match: 3D Point Cloud Matching with Smoothed Densities](https://mlsanity.com/paper/1811.06879/) (2018)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences](https://mlsanity.com/paper/2005.01014/) (2020)
- [Learning Compact Geometric Features](https://mlsanity.com/paper/1709.05056/) (2017)
- [IMLS-SLAM: scan-to-model matching based on 3D data](https://mlsanity.com/paper/1802.08633/) (2018)
- [SegICP: Integrated Deep Semantic Segmentation and Pose Estimation](https://mlsanity.com/paper/1703.01661/) (2017)
- [The Differentiable Cross-Entropy Method](https://mlsanity.com/paper/1909.12830/) (2019)
- [Mapping in a cycle: Sinkhorn regularized unsupervised learning for point cloud shapes](https://mlsanity.com/paper/2007.09594/) (2020)
- [Constrained Model-based Reinforcement Learning with Robust Cross-Entropy Method](https://mlsanity.com/paper/2010.07968/) (2020)
- [Unsupervised Partial Point Set Registration via Joint Shape Completion and Registration](https://mlsanity.com/paper/2009.05290/) (2020)
- [Cross-Entropy Method Variants for Optimization](https://mlsanity.com/paper/2009.09043/) (2020)

## Built on by (most cited first, 7 of 7)

- [Robust Outlier Rejection for 3D Registration with Variational Bayes](https://mlsanity.com/paper/2304.01514/) (2023)
- [Sampling Network Guided Cross-Entropy Method for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2109.06619/) (2021)
- [Reliable Inlier Evaluation for Unsupervised Point Cloud Registration](https://mlsanity.com/paper/2202.11292/) (2022)
- [FUSER: Feed-Forward MUltiview 3D Registration Transformer and SE(3)N Diffusion Refinement](https://mlsanity.com/paper/2512.09373/) (2025)
- [Point Cloud Registration-Driven Robust Feature Matching for 3D Siamese Object Tracking](https://mlsanity.com/paper/2209.06395/) (2022)
- [3D Registration in 30 Years: A Survey](https://mlsanity.com/paper/2412.13735/) (2024)
- [DIPR: Efficient Point Cloud Registration via Dynamic Iteration](https://mlsanity.com/paper/2312.02877/) (2023)

## Abstract (arXiv)

Point cloud registration is a fundamental problem in 3D computer vision. In this paper, we cast point cloud registration into a planning problem in reinforcement learning, which can seek the transformation between the source and target point clouds through trial and error. By modeling the point cloud registration process as a Markov decision process (MDP), we develop a latent dynamic model of point clouds, consisting of a transformation network and evaluation network. The transformation network aims to predict the new transformed feature of the point cloud after performing a rigid transformation (i.e., action) on it while the evaluation network aims to predict the alignment precision between the transformed source point cloud and target point cloud as the reward signal. Once the dynamic model of the point cloud is trained, we employ the cross-entropy method (CEM) to iteratively update the planning policy by maximizing the rewards in the point cloud registration process. Thus, the optimal policy, i.e., the transformation between the source and target point clouds, can be obtained via gradually narrowing the search space of the transformation. Experimental results on ModelNet40 and 7Scene benchmark datasets demonstrate that our method can yield good registration performance in an unsupervised manner.