# Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction

arXiv: https://arxiv.org/abs/1804.06032 · HTML: https://mlsanity.com/paper/1804.06032/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2018.

## Built on by (most cited first, 27 of 27)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [Explicit Correspondence Matching for Generalizable Neural Radiance Fields](https://mlsanity.com/paper/2304.12294/) (2023)
- [ZeroShape: Regression-based Zero-shot Shape Reconstruction](https://mlsanity.com/paper/2312.14198/) (2023)
- [Representing 3D Shapes with Probabilistic Directed Distance Fields](https://mlsanity.com/paper/2112.05300/) (2021)
- [SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling](https://mlsanity.com/paper/2208.06880/) (2022)
- [Fostering Generalization in Single-view 3D Reconstruction by Learning a Hierarchy of Local and Global Shape Priors](https://mlsanity.com/paper/2104.00476/) (2021)
- [ADOP: Approximate Differentiable One-Pixel Point Rendering](https://mlsanity.com/paper/2110.06635/) (2021)
- [Shadows Shed Light on 3D Objects](https://mlsanity.com/paper/2206.08990/) (2022)
- [Ray-ONet: Efficient 3D Reconstruction From A Single RGB Image](https://mlsanity.com/paper/2107.01899/) (2021)
- [PanopticNDT: Efficient and Robust Panoptic Mapping](https://mlsanity.com/paper/2309.13635/) (2023)
- [Shape Completion with Prediction of Uncertain Regions](https://mlsanity.com/paper/2308.00377/) (2023)
- [PyTorchGeoNodes: Enabling Differentiable Shape Programs for 3D Shape Reconstruction](https://mlsanity.com/paper/2404.10620/) (2024)
- [wildNeRF: Complete view synthesis of in-the-wild dynamic scenes captured using sparse monocular data](https://mlsanity.com/paper/2209.10399/) (2022)
- [Real-time Simultaneous Multi-Object 3D Shape Reconstruction, 6DoF Pose Estimation and Dense Grasp Prediction](https://mlsanity.com/paper/2305.09510/) (2023)
- [Enhancing 2D Representation Learning with a 3D Prior](https://mlsanity.com/paper/2406.02535/) (2024)
- [Learning Local Pattern Modularization for Point Cloud Reconstruction from Unseen Classes](https://mlsanity.com/paper/2408.14279/) (2024)
- [The Surprising Positive Knowledge Transfer in Continual 3D Object Shape Reconstruction](https://mlsanity.com/paper/2101.07295/) (2021)
- [RefiNeRF: Modelling dynamic neural radiance fields with inconsistent or missing camera parameters](https://mlsanity.com/paper/2303.08695/) (2023)
- [Evaluating Latent Generative Paradigms for High-Fidelity 3D Shape Completion from a Single Depth Image](https://mlsanity.com/paper/2511.11074/) (2025)
- [A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks](https://mlsanity.com/paper/2111.15158/) (2021)
- [Spotlights: Probing Shapes from Spherical Viewpoints](https://mlsanity.com/paper/2205.12564/) (2022)
- [Recovering Detail in 3D Shapes Using Disparity Maps](https://mlsanity.com/paper/2207.00182/) (2022)
- [MiShape: 3D Shape Modelling of Mitochondria in Microscopy](https://mlsanity.com/paper/2303.01546/) (2023)
- [Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image](https://mlsanity.com/paper/2304.12455/) (2023)
- [Pix2Repair: Implicit Shape Restoration from Images](https://mlsanity.com/paper/2305.18273/) (2023)
- [A Fusion of Variational Distribution Priors and Saliency Map Replay for Continual 3D Reconstruction](https://mlsanity.com/paper/2308.08812/) (2023)
- [Probabilistic Directed Distance Fields for Ray-Based Shape Representations](https://mlsanity.com/paper/2404.09081/) (2024)

## Abstract (arXiv)

The goal of this paper is to compare surface-based and volumetric 3D object shape representations, as well as viewer-centered and object-centered reference frames for single-view 3D shape prediction. We propose a new algorithm for predicting depth maps from multiple viewpoints, with a single depth or RGB image as input. By modifying the network and the way models are evaluated, we can directly compare the merits of voxels vs. surfaces and viewer-centered vs. object-centered for familiar vs. unfamiliar objects, as predicted from RGB or depth images. Among our findings, we show that surface-based methods outperform voxel representations for objects from novel classes and produce higher resolution outputs. We also find that using viewer-centered coordinates is advantageous for novel objects, while object-centered representations are better for more familiar objects. Interestingly, the coordinate frame significantly affects the shape representation learned, with object-centered placing more importance on implicitly recognizing the object category and viewer-centered producing shape representations with less dependence on category recognition.