mlsanityData to 7 Oct 2026

DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation

arXiv 2306.12422 ↗First posted 21 Jun 2023cs.CV cs.GR cs.LGcited by 81

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Text-to-image diffusion models pre-trained on billions of image-text pairs have recently enabled 3D content creation by optimizing a randomly initialized differentiable 3D representation with score distillation. However, the optimization process suffers slow convergence and the resultant 3D models often exhibit two limitations: (a) quality concerns such as missing attributes and distorted shape and texture; (b) extremely low diversity comparing to text-guided image synthesis. In this paper, we show that the conflict between the 3D optimization process and uniform timestep sampling in score distillation is the main reason for these limitations. To resolve this conflict, we propose to prioritize timestep sampling with monotonically non-increasing functions, which aligns the 3D optimization process with the sampling process of diffusion model. Extensive experiments show that our simple redesign significantly improves 3D content creation with faster convergence, better quality and diversity.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

33 of 39 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  2. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  3. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  6. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  7. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  8. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  10. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  11. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
13 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  5. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  6. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  7. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  8. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  9. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  10. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  11. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  12. 2023AvatarCraft: Transforming Text into Neural Human Avatars with Parameterized Shape and Pose Controlcited by 84
  13. 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 32; 2025: 18; 2026: 7.

81 papers.

Sort:

By area:

  1. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2023Adversarial Diffusion Distillationcited by 741
  4. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  6. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  7. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  8. 2023Text-to-3D with Classifier Score Distillationcited by 127
  9. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  10. 2023Noise-Free Score Distillationcited by 99
  11. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  12. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priorscited by 72
38 more
  1. 2024DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Samplingcited by 72
  2. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  3. 2023Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Promptscited by 51
  4. 2024VideoMV: Consistent Multi-View Generation Based on Large Video Generative Modelcited by 45
  5. 2024Cycle3D: High-quality and Consistent Image-to-3D Generation via Generation-Reconstruction Cyclecited by 36
  6. 2023Stable Score Distillation for High-Quality 3D Generationcited by 27
  7. 2023TOSS:High-quality Text-guided Novel View Synthesis from a Single Imagecited by 26
  8. 2024DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flowcited by 26
  9. 2023Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repaintingcited by 25
  10. 2023SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identitycited by 23
  11. 2024InTeX: Interactive Text-to-texture Synthesis via Unified Depth-aware Inpaintingcited by 22
  12. 2024Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabricationcited by 19
  13. 2023DreamComposer: Controllable 3D Object Generation via Multi-View Conditionscited by 18
  14. 2023Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clippingcited by 17
  15. 2024Urban Architect: Steerable 3D Urban Scene Generation with Layout Priorcited by 17
  16. 2023Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Modelscited by 16
  17. 2024A Survey On Text-to-3D Contents Generation In The Wildcited by 13
  18. 2024STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargetingcited by 13
  19. 2024DreamCatalyst: Fast and High-Quality 3D Editing via Controlling Editability and Identity Preservationcited by 13
  20. 2024Envision3D: One Image to 3D with Anchor Views Interpolationcited by 12
  21. 2024DreamWaltz-G: Expressive 3D Gaussian Avatars from Skeleton-Guided 2D Diffusioncited by 12
  22. 2024SIGNeRF: Scene Integrated Generation for Neural Radiance Fieldscited by 11
  23. 2024JointDreamer: Ensuring Geometry Consistency and Text Congruence in Text-to-3D Generation via Joint Score Distillationcited by 11
  24. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  25. 2024Graph Canvas for Controllable 3D Scene Generationcited by 11
  26. 2023LucidDreaming: Controllable Object-Centric 3D Generationcited by 10
  27. 2025Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objectscited by 10
  28. 2024DreamView: Injecting View-specific Text Guidance into Text-to-3D Generationcited by 9
  29. 2024Diff3DS: Generating View-Consistent 3D Sketch via Differentiable Curve Renderingcited by 9
  30. 2023Progressive Text-to-3D Generation for Automatic 3D Prototypingcited by 8
  31. 2023MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generationcited by 8
  32. 2025StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spacescited by 8
  33. 2025Wonder3D++: Cross-domain Diffusion for High-fidelity 3D Generation from a Single Imagecited by 8
  34. 2023Boosting3D: High-Fidelity Image-to-3D by Boosting 2D Diffusion Prior to 3D Prior with Progressive Learningcited by 7
  35. 2023ET3D: Efficient Text-to-3D Generation via Multi-View Distillationcited by 6
  36. 2024Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Datacited by 5
  37. 2025SegmentDreamer: Towards High-fidelity Text-to-3D Synthesis with Segmented Consistency Trajectory Distillationcited by 5
  38. 2025Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Modelscited by 5

50 most cited of 81 All 81 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024BoostDream: Efficient Refining for High-Quality Text-to-3D Generation from Multi-View Diffusion
  2. 2024DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mapping
  3. 2024VividDreamer: Towards High-Fidelity and Efficient Text-to-3D Generation
  4. 2024Flow Score Distillation for Diverse Text-to-3D Generation
  5. 2024Diffusion2: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models
  6. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance
  7. 2023X-Dreamer: Creating High-quality 3D Content by Bridging the Domain Gap Between Text-to-2D and Text-to-3D Generation
  8. 2025Bridging Geometry-Coherent Text-to-3D Generation with Multi-View Diffusion Priors and Gaussian Splatting