mlsanityData to 7 Oct 2026

DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation

arXiv 2501.16764 ↗First posted 28 Jan 2025cs.CVcited by 49

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recent advancements in 3D content generation from text or a single image struggle with limited high-quality 3D datasets and inconsistency from 2D multi-view generation. We introduce DiffSplat, a novel 3D generative framework that natively generates 3D Gaussian splats by taming large-scale text-to-image diffusion models. It differs from previous 3D generative models by effectively utilizing web-scale 2D priors while maintaining 3D consistency in a unified model. To bootstrap the training, a lightweight reconstruction model is proposed to instantly produce multi-view Gaussian splat grids for scalable dataset curation. In conjunction with the regular diffusion loss on these grids, a 3D rendering loss is introduced to facilitate 3D coherence across arbitrary views. The compatibility with image diffusion models enables seamless adaptions of numerous techniques for image generation to the 3D realm. Extensive experiments reveal the superiority of DiffSplat in text- and image-conditioned generation tasks and downstream applications. Thorough ablation studies validate the efficacy of each critical design choice and provide insights into the underlying mechanism.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 86 of its 94 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

86 of 94 references.

Closest ancestors · 31

Same direction, newest first

  1. 2024SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Views19 Aug
  2. 2024RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models9 Jul
  3. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models18 Jun
  4. 2024DIRECT-3D: Learning Direct Text-to-3D Generation on Massive Noisy 3D Data6 Jun
  5. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  6. 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refiner23 May
  7. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformer23 May
  8. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
  9. 2024VFusion3D: Learning Scalable 3D Generative Models from Video Diffusion Models18 Mar
  10. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar
  11. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  12. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors4 Mar
19 more
  1. 2024Denoising Diffusion via Image-Based Rendering5 Feb
  2. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation4 Dec
  3. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation2 Dec
  4. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  5. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion14 Nov
  6. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  7. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  8. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  9. 2023T3Bench: Benchmarking Current Progress in Text-to-3D Generation4 Oct
  10. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  11. 2023Large-Vocabulary 3D Diffusion Model with Transformer14 Sep
  12. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  13. 2023HoloFusion: Towards Photo-realistic 3D Generative Modeling28 Aug
  14. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation4 Jul
  15. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data13 Jun
  16. 2023LDM3D: Latent Diffusion Model for 3D18 May
  17. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Images29 Mar
  18. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  19. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation17 Nov

Other references · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  11. 2022Flow Matching for Generative Modelingcited by 5,426
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
38 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  7. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  8. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  9. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  10. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  11. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  12. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  13. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  14. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  15. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  16. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  17. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  18. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  19. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
  20. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  21. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
  22. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  23. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  24. 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 298
  25. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  26. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  27. 2024TripoSR: Fast 3D Object Reconstruction from a Single Imagecited by 227
  28. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  29. 2023DMV3D: Denoising Multi-View Diffusion using 3D Large Reconstruction Modelcited by 213
  30. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  31. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  32. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 138
  33. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  34. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  35. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  36. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  37. 2024SPAD : Spatially Aware Multiview Diffuserscited by 62
  38. 2024HumanSplat: Generalizable Single-Image Human Gaussian Splatting with Structure Priorscited by 54

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 32; 2026: 17.

49 papers.

Sort:

By area:

  1. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
  2. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  3. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 31
  4. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstructioncited by 24
  5. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 23
  6. 2025HiScene: Creating Hierarchical 3D Scenes with Isometric View Generationcited by 18
  7. 2025VideoRFSplat: Direct Scene-Level Text-to-3D Gaussian Splatting Generation with Flexible Pose and Multi-View Joint Modelingcited by 12
  8. 2025SIGMAN:Scaling 3D Human Gaussian Generation with Millions of Assetscited by 12
  9. 2025Nabla-R2D3: Effective and Efficient 3D Diffusion Alignment with 2D Rewardscited by 11
  10. 20253D-LATTE: Latent Space 3D Editing from Textual Instructionscited by 10
  11. 2025Diverse Text-to-Image Generation via Contrastive Noise Optimizationcited by 10
  12. 2025Text-to-3D by Stitching a Multi-view Reconstruction Network to a Video Generatorcited by 10
37 more
  1. 2025Towards Scalable Spatial Intelligence via 2D-to-3D Data Liftingcited by 8
  2. 2025Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generationcited by 7
  3. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  4. 2025Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Modelscited by 7
  5. 2025DiMeR: Disentangled Mesh Reconstruction Modelcited by 6
  6. 2025OmniGen: Unified Multimodal Sensor Generation for Autonomous Drivingcited by 5
  7. 2026Learning Physics-Grounded 4D Dynamics with Neural Gaussian Force Fieldscited by 5
  8. 2025ViDAR: Video Diffusion-Aware 4D Reconstruction From Monocular Inputscited by 4
  9. 2025VRsketch2Gaussian: 3D VR Sketch Guided 3D Object Generation with Gaussian Splattingcited by 3
  10. 2025MADrive: Memory-Augmented Driving Scene Modelingcited by 3
  11. 2025Walking the Schr"odinger Bridge: A Direct Trajectory for Text-to-3D Generationcited by 3
  12. 2025TRIM: Scalable 3D Gaussian Diffusion Inference with Temporal and Spatial Trimmingcited by 3
  13. 2026RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Spacecited by 3
  14. 2025Complete Gaussian Splats from a Single Image with Denoising Diffusion Modelscited by 2
  15. 2025Learning Unified Representation of 3D Gaussian Splattingcited by 2
  16. 2025SplatSuRe: Selective Super-Resolution for Multi-view Consistent 3D Gaussian Splattingcited by 2
  17. 2026Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstructioncited by 2
  18. 2026PixGS: Pixel-Space Diffusion for Direct 3D Gaussian Splat Generationcited by 2
  19. 2025Object Reconstruction under Occlusion with Generative Priors and Contact-induced Constraintscited by 1
  20. 2025Photo3D: Advancing Photorealistic 3D Generation through Structure-Aligned Detail Enhancementcited by 1
  21. 2026Feed-forward Gaussian Registration for Head Avatar Creation and Editingcited by 1
  22. 2026GaussianGrow: Geometry-aware Gaussian Growing from 3D Point Clouds with Text Guidancecited by 1
  23. 2026GenSplatCodec: Feed-Forward Gaussian Splatting Compression via One-Step Diffusioncited by 1
  24. 2026GS-Voxel: Fitting-Free Structured Latents for Large-Scale 3DGS Generationcited by 1
  25. 2026Sparse auto-regressive modeling for scene generation from multi-view imagescited by 1
  26. 2025DreamLifting: A Plug-in Module Lifting MV Diffusion Models for 3D Asset Generationcited by 0
  27. 2025HumanCrafter: Synergizing Generalizable Human Reconstruction and Semantic 3D Segmentationcited by 0
  28. 2025Gaussian Blending: Rethinking Alpha Blending in 3D Gaussian Splattingcited by 0
  29. 2025InStyle: Instant Appearance Stylization of 3D Shapescited by 0
  30. 2025LoGoColor: Local-Global 3D Colorization for 360deg Scenescited by 0
  31. 2026WebSplatter: Enabling Cross-Device Efficient Gaussian Splatting in Web Browsers via WebGPUcited by 0
  32. 2026AdaptiveSplat:Texture Aware Controllable 3D Gaussian Allocation for Feed-Forward Reconstructioncited by 0
  33. 2026STEREOFLOW: Progressive Stereo Matching with StereoDiT and Transition Flow Matchingcited by 0
  34. 2026Seeing the Unseen: Semantic-in-Gaussian for Sparse-View 3D Generalizationcited by 0
  35. 2026ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Viewscited by 0
  36. 2026OREO: Fidelity Alignment in 3D Generation via On-the-fly Rendering-Editing Optimizationcited by 0
  37. 2026PartiCam: Camera Controlled Video Generation with Reward Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative Gaussian Splatting: Generating 3D Scenes with Video Diffusion Priors
  2. 2026GIFSplat: Generative Prior-Guided Iterative Feed-Forward 3D Gaussian Splatting from Sparse Views
  3. 2025Zero-1-to-G: Taming Pretrained 2D Diffusion Model for Direct 3D Generation
  4. 2024DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models
  5. 2024GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstruction
  6. 2024DiffGS: Functional Gaussian Splatting Diffusion
  7. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesis
  8. 2024Enhancing Single Image to 3D Generation using Gaussian Splatting and Hybrid Diffusion Priors