mlsanityData to 7 Oct 2026

Accelerating 3D Deep Learning with PyTorch3D

arXiv 2007.08501 ↗First posted 16 Jul 2020cs.CV cs.GR cs.LGcited by 781

Not on the map. Why?

Cited by 64 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning has significantly improved 2D image recognition. Extending into 3D may advance many new applications including autonomous vehicles, virtual and augmented reality, authoring 3D content, and even improving 2D recognition. However despite growing interest, 3D deep learning remains relatively underexplored. We believe that some of this disparity is due to the engineering challenges involved in 3D deep learning, such as efficiently processing heterogeneous data and reframing graphics operations to be differentiable. We address these challenges by introducing PyTorch3D, a library of modular, efficient, and differentiable operators for 3D deep learning. It includes a fast, modular differentiable renderer for meshes and point clouds, enabling analysis-by-synthesis approaches. Compared with other differentiable renderers, PyTorch3D is more modular and efficient, allowing users to more easily extend it while also gracefully scaling to large meshes and images. We compare the PyTorch3D operators and renderer with other implementations and demonstrate significant speed and memory improvements. We also use PyTorch3D to improve the state-of-the-art for unsupervised 3D mesh and point cloud prediction from 2D images on ShapeNet. PyTorch3D is open-source and we hope it will help accelerate research in 3D deep learning.

Key facts. Cited by 781 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 101; 2022: 143; 2023: 182; 2024: 143; 2025: 128; 2026: 84.

781 papers.

Sort:

By area:

  1. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  2. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  3. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
  4. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  5. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  6. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  7. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  8. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  9. 2021Advances in Neural Renderingcited by 365
  10. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  11. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  12. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
38 more
  1. 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
  2. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  3. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  4. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  5. 2021Neural Head Avatars from Monocular RGB Videoscited by 212
  6. 2022ECON: Explicit Clothed humans Optimized via Normal integrationcited by 199
  7. 2021Shape As Points: A Differentiable Poisson Solvercited by 186
  8. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  9. 2022PointAvatar: Deformable Point-based Head Avatars from Videoscited by 183
  10. 2022Towards Metrical Reconstruction of Human Facescited by 165
  11. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  12. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  13. 2023HUGS: Human Gaussian Splatscited by 154
  14. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  15. 2023Multi3DRefer: Grounding Text Description to Multiple 3D Objectscited by 142
  16. 2021Collaborative Regression of Expressive Bodies using Moderationcited by 137
  17. 2021NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wildcited by 136
  18. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  19. 2024RVT-2: Learning Precise Manipulation from Few Demonstrationscited by 135
  20. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  21. 2024Rethinking Inductive Biases for Surface Normal Estimationcited by 122
  22. 2023StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Viewscited by 120
  23. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  24. 2022PartSLIP: Low-Shot Part Segmentation for 3D Point Clouds via Pretrained Image-Language Modelscited by 117
  25. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  26. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  27. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 107
  28. 2023Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinementcited by 106
  29. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  30. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  31. 2021gradSim: Differentiable simulation for system identification and visuomotor controlcited by 100
  32. 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
  33. 20234K4D: Real-Time 4D View Synthesis at 4K Resolutioncited by 91
  34. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  35. 2023NerfAcc: Efficient Sampling Accelerates NeRFscited by 90
  36. 2021Generative Models as Distributions of Functionscited by 89
  37. 2024Expressive Whole-Body 3D Gaussian Avatarcited by 89
  38. 2022Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldscited by 85

50 most cited of 781 All 781 (JSON)