# Consistent Two-Flow Network for Tele-Registration of Point Clouds

arXiv: https://arxiv.org/abs/2106.00329 · HTML: https://mlsanity.com/paper/2106.00329/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 38 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [PointNetLK: Robust & Efficient Point Cloud Registration using PointNet](https://mlsanity.com/paper/1903.05711/) (2019)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [RPM-Net: Robust Point Matching using Learned Features](https://mlsanity.com/paper/2003.13479/) (2020)
- [PRNet: Self-Supervised Learning for Partial-to-Partial Registration](https://mlsanity.com/paper/1910.12240/) (2019)
- [PF-Net: Point Fractal Network for 3D Point Cloud Completion](https://mlsanity.com/paper/2003.00410/) (2020)
- [Morphing and Sampling Network for Dense Point Cloud Completion](https://mlsanity.com/paper/1912.00280/) (2019)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [Cascaded Refinement Network for Point Cloud Completion](https://mlsanity.com/paper/2004.03327/) (2020)
- [3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration](https://mlsanity.com/paper/1807.09413/) (2018)
- [Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views](https://mlsanity.com/paper/1505.05641/) (2015)
- [A Large Dataset of Object Scans](https://mlsanity.com/paper/1602.02481/) (2016)
- [Unsupervised cycle-consistent deformation for shape matching](https://mlsanity.com/paper/1907.03165/) (2019)
- [Extreme Relative Pose Estimation for RGB-D Scans via Scene Completion](https://mlsanity.com/paper/1901.00063/) (2018)
- [Extreme Relative Pose Network under Hybrid Representations](https://mlsanity.com/paper/1912.11695/) (2019)
- [ALIGNet: Partial-Shape Agnostic Alignment via Unsupervised Learning](https://mlsanity.com/paper/1804.08497/) (2018)

## Built on by (most cited first, 5 of 5)

- [PU-Flow: a Point Cloud Upsampling Network with Normalizing Flows](https://mlsanity.com/paper/2107.05893/) (2021)
- [ImLoveNet: Misaligned Image-supported Registration Network for Low-overlap Point Cloud Pairs](https://mlsanity.com/paper/2207.00826/) (2022)
- [WSDesc: Weakly Supervised 3D Local Descriptor Learning for Point Cloud Registration](https://mlsanity.com/paper/2108.02740/) (2021)
- [UTOPIC: Uncertainty-aware Overlap Prediction Network for Partial Point Cloud Registration](https://mlsanity.com/paper/2208.02712/) (2022)
- [Deep Confidence Guided Distance for 3D Partial Shape Registration](https://mlsanity.com/paper/2201.11379/) (2022)

## Abstract (arXiv)

Rigid registration of partial observations is a fundamental problem in various applied fields. In computer graphics, special attention has been given to the registration between two partial point clouds generated by scanning devices. State-of-the-art registration techniques still struggle when the overlap region between the two point clouds is small, and completely fail if there is no overlap between the scan pairs. In this paper, we present a learning-based technique that alleviates this problem, and allows registration between point clouds, presented in arbitrary poses, and having little or even no overlap, a setting that has been referred to as tele-registration. Our technique is based on a novel neural network design that learns a prior of a class of shapes and can complete a partial shape. The key idea is combining the registration and completion tasks in a way that reinforces each other. In particular, we simultaneously train the registration network and completion network using two coupled flows, one that register-and-complete, and one that complete-and-register, and encourage the two flows to produce a consistent result. We show that, compared with each separate flow, this two-flow training leads to robust and reliable tele-registration, and hence to a better point cloud prediction that completes the registered scans. It is also worth mentioning that each of the components in our neural network outperforms state-of-the-art methods in both completion and registration. We further analyze our network with several ablation studies and demonstrate its performance on a large number of partial point clouds, both synthetic and real-world, that have only small or no overlap.