# 3D Pose Estimation and 3D Model Retrieval for Objects in the Wild

arXiv: https://arxiv.org/abs/1803.11493 · HTML: https://mlsanity.com/paper/1803.11493/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2018.

## Built on by (most cited first, 30 of 31)

- [SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation](https://mlsanity.com/paper/2108.08367/) (2021)
- [Zero-Shot Category-Level Object Pose Estimation](https://mlsanity.com/paper/2204.03635/) (2022)
- [NOPE: Novel Object Pose Estimation from a Single Image](https://mlsanity.com/paper/2303.13612/) (2023)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [CPPF: Towards Robust Category-Level 9D Pose Estimation in the Wild](https://mlsanity.com/paper/2203.03089/) (2022)
- [Densely Constrained Depth Estimator for Monocular 3D Object Detection](https://mlsanity.com/paper/2207.10047/) (2022)
- [Adaptive Spot-Guided Transformer for Consistent Local Feature Matching](https://mlsanity.com/paper/2303.16624/) (2023)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [Visual Localization using Imperfect 3D Models from the Internet](https://mlsanity.com/paper/2304.05947/) (2023)
- [Monte Carlo Scene Search for 3D Scene Understanding](https://mlsanity.com/paper/2103.07969/) (2021)
- [PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning](https://mlsanity.com/paper/2105.05643/) (2021)
- [Looking 3D: Anomaly Detection with 2D-3D Alignment](https://mlsanity.com/paper/2406.19393/) (2024)
- [Estimating 3D Motion and Forces of Human-Object Interactions from Internet Videos](https://mlsanity.com/paper/2111.01591/) (2021)
- [Automatically Annotating Indoor Images with CAD Models via RGB-D Scans](https://mlsanity.com/paper/2212.11796/) (2022)
- [Structured Epipolar Matcher for Local Feature Matching](https://mlsanity.com/paper/2303.16646/) (2023)
- [You Only Look at One: Category-Level Object Representations for Pose Estimation From a Single Example](https://mlsanity.com/paper/2305.12626/) (2023)
- [Structure-Aware 3D VR Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.09043/) (2022)
- [CORSAIR: Convolutional Object Retrieval and Symmetry-AIded Registration](https://mlsanity.com/paper/2103.06911/) (2021)
- [Semi-supervised Viewpoint Estimation with Geometry-aware Conditional Generation](https://mlsanity.com/paper/2104.01103/) (2021)
- [Fine-Grained VR Sketching: Dataset and Insights](https://mlsanity.com/paper/2209.10008/) (2022)
- [Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos](https://mlsanity.com/paper/2407.04384/) (2024)
- [Towards 3D VR-Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.10020/) (2022)
- [Denoising Multi-Similarity Formulation: A Self-paced Curriculum-Driven Approach for Robust Metric Learning](https://mlsanity.com/paper/2211.11751/) (2022)
- [ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation](https://mlsanity.com/paper/2212.00435/) (2022)
- [HOC-Search: Efficient CAD Model and Pose Retrieval from RGB-D Scans](https://mlsanity.com/paper/2309.06107/) (2023)
- [3D-Augmented Contrastive Knowledge Distillation for Image-based Object Pose Estimation](https://mlsanity.com/paper/2206.02531/) (2022)
- [Pose Estimation and 3D Reconstruction of Vehicles from Stereo-Images Using a Subcategory-Aware Shape Prior](https://mlsanity.com/paper/2107.10898/) (2021)
- [Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data](https://mlsanity.com/paper/2108.04091/) (2021)
- [Accidental Turntables: Learning 3D Pose by Watching Objects Turn](https://mlsanity.com/paper/2212.06300/) (2022)
- [GIMS: Image Matching System Based on Adaptive Graph Construction and Graph Neural Network](https://mlsanity.com/paper/2412.18221/) (2024)

## Abstract (arXiv)

We propose a scalable, efficient and accurate approach to retrieve 3D models for objects in the wild. Our contribution is twofold. We first present a 3D pose estimation approach for object categories which significantly outperforms the state-of-the-art on Pascal3D+. Second, we use the estimated pose as a prior to retrieve 3D models which accurately represent the geometry of objects in RGB images. For this purpose, we render depth images from 3D models under our predicted pose and match learned image descriptors of RGB images against those of rendered depth images using a CNN-based multi-view metric learning approach. In this way, we are the first to report quantitative results for 3D model retrieval on Pascal3D+, where our method chooses the same models as human annotators for 50% of the validation images on average. In addition, we show that our method, which was trained purely on Pascal3D+, retrieves rich and accurate 3D models from ShapeNet given RGB images of objects in the wild.