mlsanityData to 7 Oct 2026

Unsupervised Learning of Shape and Pose with Differentiable Point Clouds

arXiv 1810.09381 ↗First posted 22 Oct 2018cs.CV cs.LGcited by 89

Not on the map. Why?

Abstract (arXiv)

We address the problem of learning accurate 3D shape and camera pose from a collection of unlabeled category-specific images. We train a convolutional network to predict both the shape and the pose from a single image by minimizing the reprojection error: given several views of an object, the projections of the predicted shapes to the predicted camera poses should match the provided views. To deal with pose ambiguity, we introduce an ensemble of pose predictors which we then distill to a single "student" model. To allow for efficient learning of high-fidelity shapes, we represent the shapes by point clouds and devise a formulation allowing for differentiable projection of these. Our experiments show that the distilled ensemble of pose predictors learns to estimate the pose accurately, while the point cloud representation allows to predict detailed shape models. The supplementary video can be found at https://www.youtube.com/watch?v=LuIGovKeo60

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 32; 2022: 20; 2023: 16; 2024: 14; 2025: 4; 2026: 3.

89 papers.

Sort:

By area:

  1. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  2. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 684
  3. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  4. 2021Advances in Neural Renderingcited by 365
  5. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  6. 2024Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splattingcited by 111
  7. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  8. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  9. 2024Advances in 3D Generation: A Surveycited by 62
  10. 2022Motion Inspired Unsupervised Perception and Prediction in Autonomous Drivingcited by 59
  11. 2022Surface Reconstruction from Point Clouds by Learning Predictive Context Priorscited by 54
  12. 2021Style-based Point Generator with Adversarial Rendering for Point Cloud Completioncited by 45
38 more
  1. 2021Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimationcited by 45
  2. 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
  3. 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
  4. 2022Approximate Differentiable Rendering with Algebraic Surfacescited by 44
  5. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  6. 2022Cross-modal Learning for Image-Guided Point Cloud Shape Completioncited by 36
  7. 2022Reconstructing Hand-Held Objects from Monocular Videocited by 35
  8. 2021Active 3D Shape Reconstruction from Vision and Touchcited by 34
  9. 2022Occlusion-Aware Self-Supervised Monocular 6D Object Pose Estimationcited by 34
  10. 2024A survey of synthetic data augmentation methods in computer visioncited by 34
  11. 2023LU-NeRF: Scene and Pose Estimation by Synchronizing Local Unposed NeRFscited by 33
  12. 2023Flexible Techniques for Differentiable Rendering with 3D Gaussianscited by 28
  13. 2021Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matchingcited by 26
  14. 2022Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstructioncited by 26
  15. 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistencycited by 26
  16. 2023CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulationcited by 26
  17. 2021Representing 3D Shapes with Probabilistic Directed Distance Fieldscited by 21
  18. 2022Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Imagescited by 21
  19. 2022Generalizable Neural Radiance Fields for Novel View Synthesis with Transformercited by 21
  20. 2021NeRD: Neural 3D Reflection Symmetry Detectorcited by 18
  21. 2021Self-Supervised Point Cloud Completion via Inpaintingcited by 16
  22. 2021NeeDrop: Self-supervised Shape Representation from Sparse Point Clouds using Needle Droppingcited by 16
  23. 2023ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistencycited by 16
  24. 2021StrobeNet: Category-Level Multiview Reconstruction of Articulated Objectscited by 14
  25. 2021Parts2Words: Learning Joint Embedding of Point Clouds and Texts by Bidirectional Matching between Parts and Wordscited by 14
  26. 2021Discovering 3D Parts from Image Collectionscited by 13
  27. 2022GenDR: A Generalized Differentiable Renderercited by 12
  28. 2023UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fieldscited by 12
  29. 2021Free-Viewpoint RGB-D Human Performance Capture and Renderingcited by 11
  30. 2022Boosting Point Clouds Rendering via Radiance Mappingcited by 11
  31. 2022Plateau-reduced Differentiable Path Tracingcited by 10
  32. 2021ADOP: Approximate Differentiable One-Pixel Point Renderingcited by 9
  33. 2022Planes vs. Chairs: Category-guided 3D shape learning without any 3D cuescited by 9
  34. 2022Multi-View Reconstruction using Signed Ray Distance Functions (SRDF)cited by 9
  35. 2023ACL-SPC: Adaptive Closed-Loop system for Self-Supervised Point Cloud Completioncited by 9
  36. 2022Artemis: Articulated Neural Pets with Appearance and Motion synthesiscited by 8
  37. 2022Leveraging Single-View Images for Unsupervised 3D Point Cloud Completioncited by 8
  38. 2024OP-Align: Object-level and Part-level Alignment for Self-supervised Category-level Articulated Object Pose Estimationcited by 8

50 most cited of 89 All 89 (JSON)