# Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images

arXiv: https://arxiv.org/abs/1712.03904 · HTML: https://mlsanity.com/paper/1712.03904/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2017.

## Built on by (most cited first, 15 of 15)

- [FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanism](https://mlsanity.com/paper/2103.07054/) (2021)
- [FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation models](https://mlsanity.com/paper/2312.00947/) (2023)
- [Perspective Flow Aggregation for Data-Limited 6D Object Pose Estimation](https://mlsanity.com/paper/2203.09836/) (2022)
- [PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning](https://mlsanity.com/paper/2105.05643/) (2021)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [Estimating 3D Motion and Forces of Human-Object Interactions from Internet Videos](https://mlsanity.com/paper/2111.01591/) (2021)
- [Efficient Virtual View Selection for 3D Hand Pose Estimation](https://mlsanity.com/paper/2203.15458/) (2022)
- [RaSim: A Range-aware High-fidelity RGB-D Data Simulation Pipeline for Real-world Applications](https://mlsanity.com/paper/2404.03962/) (2024)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [For A More Comprehensive Evaluation of 6DoF Object Pose Tracking](https://mlsanity.com/paper/2309.07796/) (2023)
- [ConvPoseCNN2: Prediction and Refinement of Dense 6D Object Poses](https://mlsanity.com/paper/2205.11124/) (2022)
- [Category-Level 6D Object Pose Estimation with Flexible Vector-Based Rotation Representation](https://mlsanity.com/paper/2212.04632/) (2022)
- [DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction](https://mlsanity.com/paper/2110.08636/) (2021)
- [Weakly Supervised Learning of Keypoints for 6D Object Pose Estimation](https://mlsanity.com/paper/2203.03498/) (2022)
- [FSID: Fully Synthetic Image Denoising via Procedural Scene Generation](https://mlsanity.com/paper/2212.03961/) (2022)

## Abstract (arXiv)

We propose a simple and efficient method for exploiting synthetic images when training a Deep Network to predict a 3D pose from an image. The ability of using synthetic images for training a Deep Network is extremely valuable as it is easy to create a virtually infinite training set made of such images, while capturing and annotating real images can be very cumbersome. However, synthetic images do not resemble real images exactly, and using them for training can result in suboptimal performance. It was recently shown that for exemplar-based approaches, it is possible to learn a mapping from the exemplar representations of real images to the exemplar representations of synthetic images. In this paper, we show that this approach is more general, and that a network can also be applied after the mapping to infer a 3D pose: At run time, given a real image of the target object, we first compute the features for the image, map them to the feature space of synthetic images, and finally use the resulting features as input to another network which predicts the 3D pose. Since this network can be trained very effectively by using synthetic images, it performs very well in practice, and inference is faster and more accurate than with an exemplar-based approach. We demonstrate our approach on the LINEMOD dataset for 3D object pose estimation from color images, and the NYU dataset for 3D hand pose estimation from depth maps. We show that it allows us to outperform the state-of-the-art on both datasets.