# A Real World Dataset for Multi-view 3D Reconstruction

arXiv: https://arxiv.org/abs/2203.11397 · HTML: https://mlsanity.com/paper/2203.11397/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 58 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 30 of 30)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objects](https://mlsanity.com/paper/1701.05498/) (2017)
- [Learning a Multi-View Stereo Machine](https://mlsanity.com/paper/1708.05375/) (2017)
- [Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations](https://mlsanity.com/paper/2012.09988/) (2020)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation](https://mlsanity.com/paper/1908.01491/) (2019)
- [A Large Dataset of Object Scans](https://mlsanity.com/paper/1602.02481/) (2016)
- [FroDO: From Detections to 3D Objects](https://mlsanity.com/paper/2005.05125/) (2020)
- [A Skeleton-bridged Deep Learning Approach for Generating Meshes of Complex Topologies from Single RGB Images](https://mlsanity.com/paper/1903.04704/) (2019)
- [Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images](https://mlsanity.com/paper/2008.07760/) (2020)
- [Front2Back: Single View 3D Shape Reconstruction via Front to Back Prediction](https://mlsanity.com/paper/1912.10589/) (2019)
- [Weakly supervised 3D Reconstruction with Adversarial Constraint](https://mlsanity.com/paper/1705.10904/) (2017)
- [MeshMVS: Multi-View Stereo Guided Mesh Reconstruction](https://mlsanity.com/paper/2010.08682/) (2020)

## Built on by (most cited first, 9 of 9)

- [SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D](https://mlsanity.com/paper/2310.02596/) (2023)
- [Advances in 3D Generation: A Survey](https://mlsanity.com/paper/2401.17807/) (2024)
- [NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations](https://mlsanity.com/paper/2306.09109/) (2023)
- [Dense RGB SLAM with Neural Implicit Maps](https://mlsanity.com/paper/2301.08930/) (2023)
- [ZeroShape: Regression-based Zero-shot Shape Reconstruction](https://mlsanity.com/paper/2312.14198/) (2023)
- [MobileBrick: Building LEGO for 3D Reconstruction on Mobile Devices](https://mlsanity.com/paper/2303.01932/) (2023)
- [OmniShape: Zero-Shot Multi-Hypothesis Shape and Pose Estimation in the Real World](https://mlsanity.com/paper/2508.03669/) (2025)
- [Advancing Digital Twin Generation Through a Novel Simulation Framework and Quantitative Benchmarking](https://mlsanity.com/paper/2602.11314/) (2026)
- [3DMPE: 3D Multi-Perspective Embedding](https://mlsanity.com/paper/2607.04898/) (2026)

## Abstract (arXiv)

We present a dataset of 998 3D models of everyday tabletop objects along with their 847,000 real world RGB and depth images. Accurate annotations of camera poses and object poses for each image are performed in a semi-automated fashion to facilitate the use of the dataset for myriad 3D applications like shape reconstruction, object pose estimation, shape retrieval etc. We primarily focus on learned multi-view 3D reconstruction due to the lack of appropriate real world benchmark for the task and demonstrate that our dataset can fill that gap. The entire annotated dataset along with the source code for the annotation tools and evaluation baselines is available at http://www.ocrtoc.org/3d-reconstruction.html.