# Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views

arXiv: https://arxiv.org/abs/1505.05641 · HTML: https://mlsanity.com/paper/1505.05641/

## Key facts

- Cited by 94 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2015.

## Built on by (most cited first, 30 of 94)

- [FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation](https://mlsanity.com/paper/2103.02242/) (2021)
- [FS6D: Few-Shot 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2203.14628/) (2022)
- [Shelf-Supervised Mesh Prediction in the Wild](https://mlsanity.com/paper/2102.06195/) (2021)
- [Implicit-PDF: Non-Parametric Representation of Probability Distributions on the Rotation Manifold](https://mlsanity.com/paper/2106.05965/) (2021)
- [Wide-Baseline Relative Camera Pose Estimation with Directional Learning](https://mlsanity.com/paper/2106.03336/) (2021)
- [X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion](https://mlsanity.com/paper/2212.03863/) (2022)
- [PAD: A Dataset and Benchmark for Pose-agnostic Anomaly Detection](https://mlsanity.com/paper/2310.07716/) (2023)
- [NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation](https://mlsanity.com/paper/2101.12378/) (2021)
- [Planar Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2103.14644/) (2021)
- [Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion](https://mlsanity.com/paper/2208.09480/) (2022)
- [MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/2309.13042/) (2023)
- [Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimation](https://mlsanity.com/paper/2203.10339/) (2022)
- [A survey of synthetic data augmentation methods in computer vision](https://mlsanity.com/paper/2403.10075/) (2024)
- [Object Pose Estimation with Statistical Guarantees: Conformal Keypoint Detection and Geometric Uncertainty Propagation](https://mlsanity.com/paper/2303.12246/) (2023)
- [Projective Manifold Gradient Layer for Deep Rotation Regression](https://mlsanity.com/paper/2110.11657/) (2021)
- [Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data](https://mlsanity.com/paper/2112.00054/) (2021)
- [Image Compositing for Segmentation of Surgical Tools without Manual Annotations](https://mlsanity.com/paper/2102.09528/) (2021)
- [Orient Anything V2: Unifying Orientation and Rotation Understanding](https://mlsanity.com/paper/2601.05573/) (2026)
- [Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimation](https://mlsanity.com/paper/2203.14531/) (2022)
- [EgoEnv: Human-centric environment representations from egocentric video](https://mlsanity.com/paper/2207.11365/) (2022)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Image](https://mlsanity.com/paper/2111.03098/) (2021)
- [PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning](https://mlsanity.com/paper/2105.05643/) (2021)
- [Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction](https://mlsanity.com/paper/2302.13926/) (2023)
- [DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection](https://mlsanity.com/paper/2206.09592/) (2022)
- [Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI](https://mlsanity.com/paper/2311.03783/) (2023)
- [Real-Time, Flight-Ready, Non-Cooperative Spacecraft Pose Estimation Using Monocular Imagery](https://mlsanity.com/paper/2101.09553/) (2021)
- [CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds](https://mlsanity.com/paper/2103.01977/) (2021)
- [A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?](https://mlsanity.com/paper/2108.11018/) (2021)
- [3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers](https://mlsanity.com/paper/2110.08861/) (2021)

## Abstract (arXiv)

Object viewpoint estimation from 2D images is an essential task in computer vision. However, two issues hinder its progress: scarcity of training data with viewpoint annotations, and a lack of powerful features. Inspired by the growing availability of 3D models, we propose a framework to address both issues by combining render-based image synthesis and CNNs. We believe that 3D models have the potential in generating a large number of images of high variation, which can be well exploited by deep CNN with a high learning capacity. Towards this goal, we propose a scalable and overfit-resistant image synthesis pipeline, together with a novel CNN specifically tailored for the viewpoint estimation task. Experimentally, we show that the viewpoint estimation from our pipeline can significantly outperform state-of-the-art methods on PASCAL 3D+ benchmark.