# Location Field Descriptors: Single Image 3D Model Retrieval in the Wild

arXiv: https://arxiv.org/abs/1908.02853 · HTML: https://mlsanity.com/paper/1908.02853/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2019.

## Built on by (most cited first, 15 of 15)

- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space](https://mlsanity.com/paper/2309.14616/) (2023)
- [SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fields](https://mlsanity.com/paper/2212.02501/) (2022)
- [Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image](https://mlsanity.com/paper/2108.09368/) (2021)
- [PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking](https://mlsanity.com/paper/2209.07589/) (2022)
- [Looking 3D: Anomaly Detection with 2D-3D Alignment](https://mlsanity.com/paper/2406.19393/) (2024)
- [DISP6D: Disentangled Implicit Shape and Pose Learning for Scalable 6D Pose Estimation](https://mlsanity.com/paper/2107.12549/) (2021)
- [Generalizing Single-View 3D Shape Retrieval to Occlusions and Unseen Objects](https://mlsanity.com/paper/2401.00405/) (2023)
- [Leveraging Geometry for Shape Estimation from a Single RGB Image](https://mlsanity.com/paper/2111.05615/) (2021)
- [Towards 3D VR-Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.10020/) (2022)
- [Accurate Instance-Level CAD Model Retrieval in a Large-Scale Database](https://mlsanity.com/paper/2207.01339/) (2022)
- [HOC-Search: Efficient CAD Model and Pose Retrieval from RGB-D Scans](https://mlsanity.com/paper/2309.06107/) (2023)
- [Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data](https://mlsanity.com/paper/2108.04091/) (2021)
- [Optimizing Multi-Modal Models for Image-Based Shape Retrieval: The Role of Pre-Alignment and Hard Contrastive Learning](https://mlsanity.com/paper/2603.06982/) (2026)
- [PASR: Pose-Aware 3D Shape Retrieval from Occluded Single Views](https://mlsanity.com/paper/2604.22658/) (2026)

## Abstract (arXiv)

We present Location Field Descriptors, a novel approach for single image 3D model retrieval in the wild. In contrast to previous methods that directly map 3D models and RGB images to an embedding space, we establish a common low-level representation in the form of location fields from which we compute pose invariant 3D shape descriptors. Location fields encode correspondences between 2D pixels and 3D surface coordinates and, thus, explicitly capture 3D shape and 3D pose information without appearance variations which are irrelevant for the task. This early fusion of 3D models and RGB images results in three main advantages: First, the bottleneck location field prediction acts as a regularizer during training. Second, major parts of the system benefit from training on a virtually infinite amount of synthetic data. Finally, the predicted location fields are visually interpretable and unblackbox the system. We evaluate our proposed approach on three challenging real-world datasets (Pix3D, Comp, and Stanford) with different object categories and significantly outperform the state-of-the-art by up to 20% absolute in multiple 3D retrieval metrics.