mlsanityData to 7 Oct 2026

Point-E: A System for Generating 3D Point Clouds from Complex Prompts

arXiv 2212.08751 ↗First posted 16 Dec 2022cs.CV cs.LGcited by 694

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

While recent work on text-conditional 3D object generation has shown promising results, the state-of-the-art methods typically require multiple GPU-hours to produce a single sample. This is in stark contrast to state-of-the-art generative image models, which produce samples in a number of seconds or minutes. In this paper, we explore an alternative method for 3D object generation which produces 3D models in only 1-2 minutes on a single GPU. Our method first generates a single synthetic view using a text-to-image diffusion model, and then produces a 3D point cloud using a second diffusion model which conditions on the generated image. While our method still falls short of the state-of-the-art in terms of sample quality, it is one to two orders of magnitude faster to sample from, offering a practical trade-off for some use cases. We release our pre-trained point cloud diffusion models, as well as evaluation code and models, at https://github.com/openai/point-e.

Key facts. Cited by 694 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 63 of its 79 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

63 of 79 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  2. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  3. 2022Novel View Synthesis with Diffusion Models6 Oct
  4. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  5. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2022Generative Adversarial Networkscited by 5,603
  12. 2017Neural Discrete Representation Learningcited by 4,650
38 more
  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  7. 2016Improved Techniques for Training GANscited by 2,938
  8. 2022Video Diffusion Modelscited by 2,280
  9. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  10. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  11. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  13. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  14. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  15. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  16. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  17. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  18. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  19. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  20. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  21. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  22. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  23. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  24. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  25. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  26. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  27. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  28. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  29. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  30. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  31. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  32. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  33. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  34. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  35. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  36. 2020Learning Gradient Fields for Shape Generationcited by 245
  37. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  38. 2021Shape As Points: A Differentiable Poisson Solvercited by 186

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 179; 2024: 263; 2025: 152; 2026: 99.

694 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  4. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  5. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  6. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  7. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  8. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  9. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  10. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  11. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  12. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
38 more
  1. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  2. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  3. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  4. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  5. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  6. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  7. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  8. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  9. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
  10. 2023Text-to-3D using Gaussian Splattingcited by 299
  11. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  12. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  13. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  14. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  15. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  16. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  17. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  18. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  19. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  20. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  21. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  22. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  23. 2023Text-To-4D Dynamic Scene Generationcited by 204
  24. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  25. 2023Point-Bind & Point-LLM: Aligning Point Cloud with Multi-modality for 3D Understanding, Generation, and Instruction Followingcited by 192
  26. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  27. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  28. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  29. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  30. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  31. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  32. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  33. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  34. 2023SAM3D: Segment Anything in 3D Scenescited by 155
  35. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
  36. 2024GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generationcited by 151
  37. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  38. 2023Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyondcited by 139

50 most cited of 694 All 694 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient and Scalable Point Cloud Generation with Sparse Point-Voxel Diffusion Models
  2. 2023Controllable Mesh Generation Through Sparse Latent Point Diffusion Models
  3. 2025MFM-point: Multi-scale Flow Matching for Point Cloud Generation
  4. 2025Guided and Unguided Conditional Diffusion Mechanisms for Structured and Semantically-Aware 3D Point Cloud Generation
  5. 2023Generating Images with 3D Annotations Using Diffusion Models
  6. 2022Deep Generative Models on 3D Representations: A Survey
  7. 2022Fast Point Cloud Generation with Straight Flows
  8. 20253D Scene Generation: A Survey