# Learning 3D Object Categories by Looking Around Them

arXiv: https://arxiv.org/abs/1705.03951 · HTML: https://mlsanity.com/paper/1705.03951/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2017.

## Built on by (most cited first, 26 of 26)

- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [Visual Geometry Grounded Deep Structure From Motion](https://mlsanity.com/paper/2312.04563/) (2023)
- [RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild](https://mlsanity.com/paper/2208.05963/) (2022)
- [RelPose++: Recovering 6D Poses from Sparse-view Observations](https://mlsanity.com/paper/2305.04926/) (2023)
- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [Unsupervised Learning of 3D Object Categories from Videos in the Wild](https://mlsanity.com/paper/2103.16552/) (2021)
- [DOVE: Learning Deformable 3D Objects by Watching Videos](https://mlsanity.com/paper/2107.10844/) (2021)
- [NeuralDiff: Segmenting 3D objects that move in egocentric videos](https://mlsanity.com/paper/2110.09936/) (2021)
- [FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modelling](https://mlsanity.com/paper/2104.08418/) (2021)
- [REACTO: Reconstructing Articulated Objects from a Single Video](https://mlsanity.com/paper/2404.11151/) (2024)
- [Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstruction](https://mlsanity.com/paper/2503.16318/) (2025)
- [Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis](https://mlsanity.com/paper/2201.01683/) (2022)
- [CoCoNets: Continuous Contrastive 3D Scene Representations](https://mlsanity.com/paper/2104.03851/) (2021)
- [Augmenting Implicit Neural Shape Representations with Explicit Deformation Fields](https://mlsanity.com/paper/2108.08931/) (2021)
- [Depth Field Networks for Generalizable Multi-view Scene Representation](https://mlsanity.com/paper/2207.14287/) (2022)
- [MoDA: Modeling Deformable 3D Objects from Casual Videos](https://mlsanity.com/paper/2304.08279/) (2023)
- [Structural Causal 3D Reconstruction](https://mlsanity.com/paper/2207.10156/) (2022)
- [Unified Semantic Transformer for 3D Scene Understanding](https://mlsanity.com/paper/2512.14364/) (2025)
- [Lifting 2D Object Locations to 3D by Discounting LiDAR Outliers across Objects and Views](https://mlsanity.com/paper/2109.07945/) (2021)
- [EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation](https://mlsanity.com/paper/2603.09385/) (2026)
- [ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation](https://mlsanity.com/paper/2212.00435/) (2022)
- [Accidental Turntables: Learning 3D Pose by Watching Objects Turn](https://mlsanity.com/paper/2212.06300/) (2022)
- [Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models](https://mlsanity.com/paper/2606.05833/) (2026)
- [Neural Camera Models](https://mlsanity.com/paper/2208.12903/) (2022)
- [Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image](https://mlsanity.com/paper/2304.12455/) (2023)

## Abstract (arXiv)

Traditional approaches for learning 3D object categories use either synthetic data or manual supervision. In this paper, we propose a method which does not require manual annotations and is instead cued by observing objects from a moving vantage point. Our system builds on two innovations: a Siamese viewpoint factorization network that robustly aligns different videos together without explicitly comparing 3D shapes; and a 3D shape completion network that can extract the full shape of an object from partial observations. We also demonstrate the benefits of configuring networks to perform probabilistic predictions as well as of geometry-aware data augmentation schemes. We obtain state-of-the-art results on publicly-available benchmarks.