# Sim2Real Instance-Level Style Transfer for 6D Pose Estimation

arXiv: https://arxiv.org/abs/2203.02069 · HTML: https://mlsanity.com/paper/2203.02069/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 32 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [SSD-6D: Making RGB-based 3D detection and 6D pose estimation great again](https://mlsanity.com/paper/1711.10006/) (2017)
- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Image to Image Translation for Domain Adaptation](https://mlsanity.com/paper/1712.00479/) (2017)
- [Implicit 3D Orientation Learning for 6D Object Detection from RGB Images](https://mlsanity.com/paper/1902.01275/) (2019)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [Self6D: Self-Supervised Monocular 6D Object Pose Estimation](https://mlsanity.com/paper/2004.06468/) (2020)
- [BlenderProc](https://mlsanity.com/paper/1911.01911/) (2019)
- [Enhancing Photorealism Enhancement](https://mlsanity.com/paper/2105.04619/) (2021)
- [Towards Instance-level Image-to-Image Translation](https://mlsanity.com/paper/1905.01744/) (2019)
- [Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement](https://mlsanity.com/paper/2003.12649/) (2020)
- [Benchmarking Domain Randomisation for Visual Sim-to-Real Transfer](https://mlsanity.com/paper/2011.07112/) (2020)
- [DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-scale Consistency](https://mlsanity.com/paper/2104.03658/) (2021)
- [Domain Transfer for 3D Pose Estimation from Color Images without Manual Annotations](https://mlsanity.com/paper/1810.03707/) (2018)
- [Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics](https://mlsanity.com/paper/2005.05659/) (2020)
- [Style-transfer GANs for bridging the domain gap in synthetic pose estimator training](https://mlsanity.com/paper/2004.13681/) (2020)

## Built on by (most cited first, 10 of 10)

- [Deep Learning-Based Object Pose Estimation: A Comprehensive Survey](https://mlsanity.com/paper/2405.07801/) (2024)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [Challenges for Monocular 6D Object Pose Estimation in Robotics](https://mlsanity.com/paper/2307.12172/) (2023)
- [GS-Pose: Category-Level Object Pose Estimation via Geometric and Semantic Correspondence](https://mlsanity.com/paper/2311.13777/) (2023)
- [Open Challenges for Monocular Single-shot 6D Object Pose Estimation](https://mlsanity.com/paper/2302.11827/) (2023)
- [6IMPOSE: Bridging the Reality Gap in 6D Pose Estimation for Robotic Grasping](https://mlsanity.com/paper/2208.14288/) (2022)
- [WALDO: Where Unseen Model-based 6D Pose Estimation Meets Occlusion](https://mlsanity.com/paper/2511.15874/) (2025)
- [Component-Aware Structure-Preserving Style Transfer for Satellite Visual Sim2Real Data Construction](https://mlsanity.com/paper/2605.19624/) (2026)
- [SCOPE: Semantic Conditioning for Sim2Real Category-Level Object Pose Estimation in Robotics](https://mlsanity.com/paper/2509.24572/) (2025)
- [End-to-end example-based sim-to-real RL policy transfer based on neural stylisation with application to robotic cutting](https://mlsanity.com/paper/2601.20846/) (2026)

## Abstract (arXiv)

In recent years, synthetic data has been widely used in the training of 6D pose estimation networks, in part because it automatically provides perfect annotation at low cost. However, there are still non-trivial domain gaps, such as differences in textures/materials, between synthetic and real data. These gaps have a measurable impact on performance. To solve this problem, we introduce a simulation to reality (sim2real) instance-level style transfer for 6D pose estimation network training. Our approach transfers the style of target objects individually, from synthetic to real, without human intervention. This improves the quality of synthetic data for training pose estimation networks. We also propose a complete pipeline from data collection to the training of a pose estimation network and conduct extensive evaluation on a real-world robotic platform. Our evaluation shows significant improvement achieved by our method in both pose estimation performance and the realism of images adapted by the style transfer.