# NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations

arXiv: https://arxiv.org/abs/2306.09109 · HTML: https://mlsanity.com/paper/2306.09109/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 73 references that are in the map.
- First posted on arXiv 15 Jun 2023.

## Builds on (most cited first, 30 of 52)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [Kubric: A scalable dataset generator](https://mlsanity.com/paper/2203.03570/) (2022)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [ABC: A Big CAD Model Dataset For Geometric Deep Learning](https://mlsanity.com/paper/1812.06216/) (2018)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)

## Built on by (most cited first, 30 of 53)

- [Probing the 3D Awareness of Visual Foundation Models](https://mlsanity.com/paper/2404.08636/) (2024)
- [Hi3DGen: High-fidelity 3D Geometry Generation from Images via Normal Bridging](https://mlsanity.com/paper/2503.22236/) (2025)
- [RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models](https://mlsanity.com/paper/2412.07679/) (2024)
- [OmniGlue: Generalizable Feature Matching with Foundation Model Guidance](https://mlsanity.com/paper/2405.12979/) (2024)
- [TIPS: Text-Image Pretraining with Spatial awareness](https://mlsanity.com/paper/2410.16512/) (2024)
- [MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors](https://mlsanity.com/paper/2409.15273/) (2024)
- [SPA: 3D Spatial-Awareness Enables Effective Embodied Representation](https://mlsanity.com/paper/2410.08208/) (2024)
- [LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models](https://mlsanity.com/paper/2504.14032/) (2025)
- [E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training](https://mlsanity.com/paper/2512.10950/) (2025)
- [SHINOBI: Shape and Illumination using Neural Object Decomposition via BRDF Optimization In-the-wild](https://mlsanity.com/paper/2401.10171/) (2024)
- [Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuning](https://mlsanity.com/paper/2411.19458/) (2024)
- [Customizing Text-to-Image Diffusion with Object Viewpoint Control](https://mlsanity.com/paper/2404.12333/) (2024)
- [Toward a Holistic Evaluation of Robustness in CLIP Models](https://mlsanity.com/paper/2410.01534/) (2024)
- [Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation](https://mlsanity.com/paper/2412.15211/) (2024)
- [TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment](https://mlsanity.com/paper/2604.12012/) (2026)
- [MVImgNet2.0: A Larger-scale Dataset of Multi-view Images](https://mlsanity.com/paper/2412.01430/) (2024)
- [UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback](https://mlsanity.com/paper/2511.01678/) (2025)
- [PACE: A Large-Scale Dataset with Pose Annotations in Cluttered Environments](https://mlsanity.com/paper/2312.15130/) (2023)
- [CAMEO: Correspondence-Attention Alignment for Multi-View Diffusion Models](https://mlsanity.com/paper/2512.03045/) (2025)
- [Efficient Universal Perception Encoder](https://mlsanity.com/paper/2603.22387/) (2026)
- [Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis](https://mlsanity.com/paper/2412.03570/) (2024)
- [MV-S2V: Multi-View Subject-Consistent Video Generation](https://mlsanity.com/paper/2601.17756/) (2026)
- [MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning](https://mlsanity.com/paper/2603.02024/) (2026)
- [ID-Pose: Sparse-view Camera Pose Estimation by Inverting Diffusion Models](https://mlsanity.com/paper/2306.17140/) (2023)
- [Learning Robust Multi-Scale Representation for Neural Radiance Fields from Unposed Images](https://mlsanity.com/paper/2311.04521/) (2023)
- [Probing the Mid-level Vision Capabilities of Self-Supervised Learning](https://mlsanity.com/paper/2411.17474/) (2024)
- [Can Generative Video Models Help Pose Estimation?](https://mlsanity.com/paper/2412.16155/) (2024)
- [Stream3D: Sequential Multi-View 3D Generation via Evidential Memory](https://mlsanity.com/paper/2605.21472/) (2026)
- [GeoBench: Benchmarking and Analyzing Monocular Geometry Estimation Models](https://mlsanity.com/paper/2406.12671/) (2024)
- [Aligned Novel View Image and Geometry Synthesis via Cross-modal Attention Instillation](https://mlsanity.com/paper/2506.11924/) (2025)

## Abstract (arXiv)

Recent advances in neural reconstruction enable high-quality 3D object reconstruction from casually captured image collections. Current techniques mostly analyze their progress on relatively simple image collections where Structure-from-Motion (SfM) techniques can provide ground-truth (GT) camera poses. We note that SfM techniques tend to fail on in-the-wild image collections such as image search results with varying backgrounds and illuminations. To enable systematic research progress on 3D reconstruction from casual image captures, we propose NAVI: a new dataset of category-agnostic image collections of objects with high-quality 3D scans along with per-image 2D-3D alignments providing near-perfect GT camera parameters. These 2D-3D alignments allow us to extract accurate derivative annotations such as dense pixel correspondences, depth and segmentation maps. We demonstrate the use of NAVI image collections on different problem settings and show that NAVI enables more thorough evaluations that were not possible with existing datasets. We believe NAVI is beneficial for systematic research progress on 3D reconstruction and correspondence estimation. Project page: https://navidataset.github.io