mlsanityData to 7 Oct 2026

Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision

arXiv 1612.00814 ↗First posted 1 Dec 2016cs.CV cs.GR cs.LGcited by 106

Not on the map. Why?

Abstract (arXiv)

Understanding the 3D world is a fundamental problem in computer vision. However, learning a good representation of 3D objects is still an open problem due to the high dimensionality of the data and many factors of variation involved. In this work, we investigate the task of single-view 3D object reconstruction from a learning agent's perspective. We formulate the learning process as an interaction between 3D and 2D representations and propose an encoder-decoder network with a novel projection loss defined by the perspective transformation. More importantly, the projection loss enables the unsupervised learning using 2D observation without explicit 3D supervision. We demonstrate the ability of the model in generating 3D volume from a single 2D image with three sets of experiments: (1) learning from single-class objects; (2) learning from multi-class objects and (3) testing on novel object classes. Results show superior performance and better generalization ability for 3D object reconstruction when the projection loss is involved.

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 36; 2022: 24; 2023: 23; 2024: 10; 2025: 10; 2026: 2.

106 papers.

Sort:

By area:

  1. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  2. 2022Kubric: A scalable dataset generatorcited by 373
  3. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  4. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 306
  5. 2021Neural Articulated Radiance Fieldcited by 196
  6. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  7. 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
  8. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  9. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  10. 2021HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputscited by 112
  11. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstructioncited by 94
  12. 2021H3D-Net: Few-Shot High-Fidelity 3D Head Reconstructioncited by 81
38 more
  1. 2022Deforming Radiance Fields with Cagescited by 78
  2. 2025RayZer: A Self-supervised Large View Synthesis Modelcited by 74
  3. 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
  4. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  5. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  6. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  7. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  8. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  9. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  10. 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
  11. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  12. 2022Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimationcited by 34
  13. 2022Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshescited by 34
  14. 2021Fast and Explicit Neural View Synthesiscited by 28
  15. 2022Neural Novel Actor: Learning a Generalized Animatable Neural Representation for Human Actorscited by 28
  16. 2018Following High-level Navigation Instructions on a Simulated Quadcopter with Imitation Learningcited by 27
  17. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  18. 2021Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matchingcited by 26
  19. 2022SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminatorscited by 26
  20. 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistencycited by 26
  21. 2021OctField: Hierarchical Implicit Functions for 3D Modelingcited by 25
  22. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  23. 2021Learning with Algorithmic Supervision via Continuous Relaxationscited by 21
  24. 2021Representing 3D Shapes with Probabilistic Directed Distance Fieldscited by 21
  25. 2024Differentiable Robot Renderingcited by 20
  26. 2021CoCoNets: Continuous Contrastive 3D Scene Representationscited by 19
  27. 2023Decoupling Dynamic Monocular Videos for Dynamic View Synthesiscited by 17
  28. 2021Self-Supervised Point Cloud Completion via Inpaintingcited by 16
  29. 2023ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistencycited by 16
  30. 2023Single-Shot Implicit Morphable Faces with Consistent Texture Parameterizationcited by 16
  31. 2024Rasterized Edge Gradients: Handling Discontinuities Differentiablycited by 14
  32. 2022Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Spacecited by 13
  33. 2022GenDR: A Generalized Differentiable Renderercited by 12
  34. 2022Structural Causal 3D Reconstructioncited by 12
  35. 2023UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fieldscited by 12
  36. 2021A Geometry-Informed Deep Learning Framework for Ultra-Sparse 3D Tomographic Image Reconstructioncited by 11
  37. 2021Multi-Head ReLU Implicit Neural Representation Networkscited by 10
  38. 2021Mask-ToF: Learning Microlens Masks for Flying Pixel Correction in Time-of-Flight Imagingcited by 9

50 most cited of 106 All 106 (JSON)