mlsanityData to 7 Oct 2026

Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation

arXiv 2412.21117 ↗First posted 30 Dec 2024cs.CVcited by 33

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

In this work, we introduce Prometheus, a 3D-aware latent diffusion model for text-to-3D generation at both object and scene levels in seconds. We formulate 3D scene generation as multi-view, feed-forward, pixel-aligned 3D Gaussian generation within the latent diffusion paradigm. To ensure generalizability, we build our model upon pre-trained text-to-image generation model with only minimal adjustments, and further train it using a large number of images from both single-view and multi-view datasets. Furthermore, we introduce an RGB-D latent space into 3D Gaussian generation to disentangle appearance and geometry information, enabling efficient feed-forward generation of 3D Gaussians with better fidelity and geometry. Extensive experimental results demonstrate the effectiveness of our method in both feed-forward 3D Gaussian reconstruction and text-to-3D generation. Project page: https://freemty.github.io/project-prometheus/

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 97 of its 112 references that are in the map. First posted on arXiv 30 Dec 2024.

Builds on

97 of 112 references.

Closest ancestors · 32

Same direction, newest first

  1. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion7 Nov
  2. 2024DiffGS: Functional Gaussian Splatting Diffusion25 Oct
  3. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstruction22 Oct
  4. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  5. 2024ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model29 Aug
  6. 2024MultiDiff: Consistent Novel View Synthesis from a Single Image26 Jun
  7. 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text25 Jun
  8. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models18 Jun
  9. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priors3 Jun
  10. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer24 May
  11. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer23 May
  12. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
20 more
  1. 2024Dual3D: Efficient and Consistent Text-to-3D Generation with Dual-mode Multi-view Latent Diffusion16 May
  2. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion10 Apr
  3. 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar
  4. 2023HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D26 Dec
  5. 2023ReconFusion: 3D Reconstruction with Diffusion Priors5 Dec
  6. 2023ViVid-1-to-3: Novel View Synthesis with Video Diffusion Models3 Dec
  7. 2023WildFusion: Learning 3D-Aware Latent Diffusion Models in View Space22 Nov
  8. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  9. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models12 Oct
  10. 2023T3Bench: Benchmarking Current Progress in Text-to-3D Generation4 Oct
  11. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  12. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  13. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  14. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  15. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  16. 2022RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Models12 Dec
  17. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  18. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation1 Dec
  19. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  20. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 65

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2023Segment Anythingcited by 8,158
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  11. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
38 more
  1. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  6. 2024Depth Anything V2cited by 1,385
  7. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  10. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  11. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  12. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  13. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  14. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  15. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  16. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  17. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  18. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  19. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  20. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  21. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  22. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  23. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  24. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  25. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  26. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
  27. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  28. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  29. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  30. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  31. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  32. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  33. 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
  34. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  35. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  36. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  37. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  38. 2023WonderJourney: Going from Anywhere to Everywherecited by 157

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 15.

33 papers.

Sort:

By area:

  1. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  2. 2025FlexWorld: Progressively Expanding 3D Scenes for Flexiable-View Synthesiscited by 37
  3. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 30
  4. 20253D Scene Generation: A Surveycited by 24
  5. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstructioncited by 24
  6. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priorscited by 19
  7. 2025A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generationcited by 19
  8. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  9. 2025GeoVideo: Introducing Geometric Regularization into Video Generation Modelcited by 14
  10. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 12
  11. 2026Repurposing Geometric Foundation Models for Multi-view Diffusioncited by 12
  12. 2025LAYOUTDREAMER: Physics-guided Layout for Text-to-3D Compositional Scene Generationcited by 11
21 more
  1. 2025WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Drivingcited by 11
  2. 20253D-LATTE: Latent Space 3D Editing from Textual Instructionscited by 10
  3. 2025Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generatorcited by 10
  4. 2025EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent Diffusioncited by 6
  5. 2025Terra: Explorable Native 3D World Model with Point Latentscited by 4
  6. 2026RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Spacecited by 3
  7. 2026Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scalecited by 2
  8. 2025PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenescited by 1
  9. 2025CustomX: Unified Character, Action, and Scene Customization in Video World Modelscited by 1
  10. 2026OneWorld: Taming Scene Generation with 3D Unified Representation Autoencodercited by 1
  11. 2026Immunizing 3D Gaussian Generative Models Against Unauthorized Fine-Tuning via Attribute-Space Trapscited by 1
  12. 2026FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matchingcited by 1
  13. 2026Crafting Your Evolving Dreams: Concept-Incremental Versatile Customizationcited by 1
  14. 2026JointEdit3D: Feed-Forward 3D Scene Editing in a Unified Latent Spacecited by 1
  15. 2025Attacks on Approximate Caches in Text-to-Image Diffusion Modelscited by 0
  16. 2026ATATA: One Algorithm to Align Them Allcited by 0
  17. 2026CGGS: Consistency-Augmented Geometric Gaussian Splatting for Ego-Centric 3D Scene Generationcited by 0
  18. 2026VaseMuseum: Digital Intelligent Museum for Ancient Greek Potterycited by 0
  19. 2026Seeing the Unseen: Semantic-in-Gaussian for Sparse-View 3D Generalizationcited by 0
  20. 2026ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Viewscited by 0
  21. 2026Monocular Depth Estimation from a Single Image: Progress and Opportunitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LT3SD: Latent Trees for 3D Scene Diffusion
  2. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors
  3. 2025Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors
  4. 2023PI3D: Efficient Text-to-3D Generation with Pseudo-Image Diffusion
  5. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis
  6. 2025Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generation
  7. 2024Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior
  8. 2024HexaGen3D: StableDiffusion is just one step away from Fast and Diverse Text-to-3D Generation