mlsanityData to 7 Oct 2026

4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling

arXiv 2311.17984 ↗First posted 29 Nov 2023cs.CVcited by 184

Not on the map. Why?

Cited by 71 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes. However, current text-to-4D methods face a three-way tradeoff between the quality of scene appearance, 3D structure, and motion. For example, text-to-image models and their 3D-aware variants are trained on internet-scale image datasets and can be used to produce scenes with realistic appearance and 3D structure -- but no motion. Text-to-video models are trained on relatively smaller video datasets and can produce scenes with motion, but poorer appearance and 3D structure. While these models have complementary strengths, they also have opposing weaknesses, making it difficult to combine them in a way that alleviates this three-way tradeoff. Here, we introduce hybrid score distillation sampling, an alternating optimization procedure that blends supervision signals from multiple pre-trained diffusion models and incorporates benefits of each for high-fidelity text-to-4D generation. Using hybrid SDS, we demonstrate synthesis of 4D scenes with compelling appearance, 3D structure, and motion.

Key facts. Cited by 184 papers in the map (data through 7 Oct 2026). Builds on 67 of its 82 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

67 of 82 references.

References · 67

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  12. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  5. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  9. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  12. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  13. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  14. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  15. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  16. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  17. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  18. 2016Generative Adversarial Text to Image Synthesiscited by 735
  19. 2023Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesiscited by 731
  20. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  21. 2023ModelScope Text-to-Video Technical Reportcited by 651
  22. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  23. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  24. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  25. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  26. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  27. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  28. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  29. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  30. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  31. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  32. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  33. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 273
  34. 2016Image Captioning with Semantic Attentioncited by 260
  35. 2021CLIP-Forge: Towards Zero-Shot Text-to-Shape Generationcited by 254
  36. 2021Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptionscited by 226
  37. 2023Text-To-4D Dynamic Scene Generationcited by 204
  38. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 70; 2025: 76; 2026: 34.

184 papers.

Sort:

By area:

  1. 2023DreamGaussian4D: Generative 4D Gaussian Splattingcited by 191
  2. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
  3. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  4. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Modelscited by 169
  5. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 160
  6. 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
  7. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  8. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  9. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  10. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  11. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 139
  12. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
38 more
  1. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 116
  2. 20244Diffusion: Multi-view Video Diffusion Model for 4D Generationcited by 104
  3. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  4. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
  5. 2024Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Modelscited by 99
  6. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 98
  7. 2025Controllable Video Generation: A Surveycited by 93
  8. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  9. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 86
  10. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  11. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  12. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  13. 2024ReCapture: Generative Video Camera Controls for User-Provided Videos using Masked Video Fine-Tuningcited by 68
  14. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Modelscited by 64
  15. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  16. 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
  17. 2024DreamMesh4D: Video-to-4D Generation with Sparse-Controlled Gaussian-Mesh Hybrid Representationcited by 58
  18. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  19. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  20. 2024CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Modelscited by 52
  21. 2024Comp4D: LLM-Guided Compositional 4D Scene Generationcited by 50
  22. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
  23. 2024EG4D: Explicit Generation of 4D Object without Score Distillationcited by 45
  24. 2025SV4D 2.0: Enhancing Spatio-Temporal Consistency in Multi-View Video Diffusion for High-Quality 4D Generationcited by 44
  25. 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
  26. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 43
  27. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 41
  28. 2024Vivid-ZOO: Multi-View Video Generation with Diffusion Modelcited by 40
  29. 2025Motion Anything: Any to Motion Generationcited by 40
  30. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  31. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 37
  32. 2024Rethinking Score Distillation as a Bridge Between Image Distributionscited by 36
  33. 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
  34. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
  35. 2025Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesiscited by 33
  36. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Predictioncited by 33
  37. 2024The Scene Language: Representing Scenes with Programs, Words, and Embeddingscited by 31
  38. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 31

50 most cited of 184 All 184 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D
  2. 2024Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior
  3. 2024Semantic Score Distillation Sampling for Compositional Text-to-3D Generation
  4. 2025Vision-Language Models as Differentiable Semantic and Spatial Rewards for Text-to-3D Generation
  5. 2025Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation
  6. 2023DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation
  7. 2024I4VGen: Image as Free Stepping Stone for Text-to-Video Generation
  8. 2024Grounded Compositional and Diverse Text-to-3D with Pretrained Multi-View Diffusion Model