# Category-Level 3D Non-Rigid Registration from Single-View RGB Images

arXiv: https://arxiv.org/abs/2008.07203 · HTML: https://mlsanity.com/paper/2008.07203/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2020.

## Built on by (most cited first, 1 of 1)

- [Learning Generalizable Tool Use with Non-rigid Grasp-pose Registration](https://mlsanity.com/paper/2307.16499/) (2023)

## Abstract (arXiv)

In this paper, we propose a novel approach to solve the 3D non-rigid registration problem from RGB images using Convolutional Neural Networks (CNNs). Our objective is to find a deformation field (typically used for transferring knowledge between instances, e.g., grasping skills) that warps a given 3D canonical model into a novel instance observed by a single-view RGB image. This is done by training a CNN that infers a deformation field for the visible parts of the canonical model and by employing a learned shape (latent) space for inferring the deformations of the occluded parts. As result of the registration, the observed model is reconstructed. Because our method does not need depth information, it can register objects that are typically hard to perceive with RGB-D sensors, e.g. with transparent or shiny surfaces. Even without depth data, our approach outperforms the Coherent Point Drift (CPD) registration method for the evaluated object categories.