mlsanityData to 7 Oct 2026

Shap-E: Generating Conditional 3D Implicit Functions

arXiv 2305.02463 ↗First posted 3 May 2023cs.CV cs.LGcited by 379

Not on the map. Why?

Cited by 116 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Shap-E, a conditional generative model for 3D assets. Unlike recent work on 3D generative models which produce a single output representation, Shap-E directly generates the parameters of implicit functions that can be rendered as both textured meshes and neural radiance fields. We train Shap-E in two stages: first, we train an encoder that deterministically maps 3D assets into the parameters of an implicit function; second, we train a conditional diffusion model on outputs of the encoder. When trained on a large dataset of paired 3D and text data, our resulting models are capable of generating complex and diverse 3D assets in a matter of seconds. When compared to Point-E, an explicit generative model over point clouds, Shap-E converges faster and reaches comparable or better sample quality despite modeling a higher-dimensional, multi-representation output space. We release model weights, inference code, and samples at https://github.com/openai/shap-e.

Key facts. Cited by 379 papers in the map (data through 7 Oct 2026). Builds on 60 of its 74 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

60 of 74 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
38 more
  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  10. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2015Variational Inference with Normalizing Flowscited by 2,161
  13. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  14. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  15. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  16. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  17. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  18. 2017Mixed Precision Trainingcited by 1,135
  19. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  20. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  21. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  22. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  23. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  24. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  25. 2020Jukebox: A Generative Model for Musiccited by 535
  26. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  27. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  28. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  29. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  30. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  31. 2023MusicLM: Generating Music From Textcited by 442
  32. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  33. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  34. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  35. 2022Novel View Synthesis with Diffusion Modelscited by 299
  36. 2022AudioGen: Textually Guided Audio Generationcited by 292
  37. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  38. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 74; 2024: 165; 2025: 87; 2026: 53.

379 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  4. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  5. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  6. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  7. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  8. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  9. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  10. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  11. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  12. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
38 more
  1. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  2. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  3. 2023Text-to-3D using Gaussian Splattingcited by 299
  4. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  5. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  6. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  7. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  8. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 245
  9. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  10. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  11. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  12. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  13. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  14. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  15. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  16. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  17. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 120
  18. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  19. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  20. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  21. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
  22. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  23. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  24. 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstructioncited by 94
  25. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  26. 2023HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splattingcited by 89
  27. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  28. 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 87
  29. 2024EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generationcited by 84
  30. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  31. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  32. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Modelscited by 79
  33. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 73
  34. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
  35. 2024DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Samplingcited by 72
  36. 2023GPT4Point: A Unified Framework for Point-Language Understanding and Generationcited by 69
  37. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  38. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65

50 most cited of 379 All 379 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 20233DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models
  2. 2022Deep Generative Models on 3D Representations: A Survey
  3. 2023Conditional Generative Modeling for Images, 3D Animations, and Video
  4. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation
  5. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
  6. 2023Breathing New Life into 3D Assets with Generative Repainting
  7. 2024GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling
  8. 2024Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation