mlsanityData to 7 Oct 2026

LT3SD: Latent Trees for 3D Scene Diffusion

arXiv 2409.08215 ↗First posted 12 Sep 2024cs.CV cs.AIcited by 53

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present LT3SD, a novel latent diffusion model for large-scale 3D scene generation. Recent advances in diffusion models have shown impressive results in 3D object generation, but are limited in spatial extent and quality when extended to 3D scenes. To generate complex and diverse 3D scene structures, we introduce a latent tree representation to effectively encode both lower-frequency geometry and higher-frequency detail in a coarse-to-fine hierarchy. We can then learn a generative diffusion process in this latent 3D scene space, modeling the latent components of a scene at each resolution level. To synthesize large-scale scenes with varying sizes, we train our diffusion model on scene patches and synthesize arbitrary-sized output 3D scenes through shared diffusion generation across multiple scene patches. Through extensive experiments, we demonstrate the efficacy and benefits of LT3SD for large-scale, high-quality unconditional 3D scene generation and for probabilistic completion for partial scene observations.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 50 of its 64 references that are in the map. First posted on arXiv 12 Sep 2024.

Builds on

50 of 64 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024SemCity: Semantic Scene Generation with Triplane Diffusion12 Mar
  2. 2023Pyramid Diffusion for Fine 3D Large Scene Generation20 Nov
  3. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models19 Apr
  4. 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction13 Apr
  5. 2023Generative Novel View Synthesis with 3D-Aware Diffusion Models5 Apr
  6. 2023DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis24 Mar
  7. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  8. 2023Diffusion Probabilistic Models for Scene-Scale 3D Categorical Data2 Jan
  9. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion12 Dec
  10. 20223D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models1 Dec
  11. 20223D Neural Field Generation using Triplane Diffusion30 Nov
  12. 2022Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions24 Nov
3 more
  1. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
  3. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  10. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  11. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  12. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
23 more
  1. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  2. 2020Neural Sparse Voxel Fieldscited by 959
  3. 2020Convolutional Occupancy Networkscited by 760
  4. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  5. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  6. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  7. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  8. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  9. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  10. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  11. 2020Learning Gradient Fields for Shape Generationcited by 245
  12. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  13. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  14. 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 183
  15. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  16. 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
  17. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  18. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  19. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  20. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  21. 2023CC3D: Layout-Conditioned Generation of Compositional 3D Scenescited by 58
  22. 2020SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scanscited by 34
  23. 2021RetrievalFuse: Neural 3D Scene Reconstruction with a Databasecited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 22; 2026: 28.

53 papers.

Sort:

By area:

  1. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  2. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  3. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  4. 2024MeshArt: Generating Articulated Meshes with Structure-Guided Transformerscited by 40
  5. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 40
  6. 2025SynCity: Training-Free Generation of 3D Worldscited by 30
  7. 2025NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenescited by 19
  8. 2025HiScene: Creating Hierarchical 3D Scenes with Isometric View Generationcited by 18
  9. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  10. 2025WorldGrow: Generating Infinite 3D Worldcited by 14
  11. 2025Orientation Matters: Making 3D Generative Models Orientation-Alignedcited by 13
  12. 2025Efficient Autoregressive Shape Generation via Octree-Based Adaptive Tokenizationcited by 12
38 more
  1. 2026SegviGen: Repurposing 3D Generative Model for Part Segmentationcited by 9
  2. 2024DNF: Unconditional 4D Generation with Dictionary-based Neural Fieldscited by 8
  3. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Datacited by 7
  4. 2026SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generationcited by 6
  5. 2026WorldClaw: Agentic 3D Open-World Generation at Scalecited by 6
  6. 2025Towards Generating Realistic 3D Semantic Training Data for Autonomous Drivingcited by 5
  7. 2025ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxiescited by 5
  8. 2026FlowScene: Style-Consistent Indoor Scene Generation with Multimodal Graph Rectified Flowcited by 4
  9. 2026Extend3D: Town-Scale 3D Generationcited by 4
  10. 2026GenAssets: Generating in-the-wild 3D Assets in Latent Spacecited by 4
  11. 2025Computer-Aided Layout Generation for Building Design: A Reviewcited by 3
  12. 2025Look Beyond: Two-Stage Scene View Generation via Panorama and Video Diffusioncited by 3
  13. 2025CymbaDiff: Structured Spatial Diffusion for Sketch-based 3D Semantic Urban Scene Generationcited by 3
  14. 2025MFM-point: Multi-scale Flow Matching for Point Cloud Generationcited by 3
  15. 2025Self-Evolving 3D Scene Generation from a Single Imagecited by 3
  16. 2026Repurposing 3D Generative Model for Autoregressive Layout Generationcited by 3
  17. 2025Constructing a 3D Scene from a Single Imagecited by 2
  18. 2026LiTo: Surface Light Field Tokenizationcited by 2
  19. 2026WorldAgents: Can Foundation Image Models be Agents for 3D World Models?cited by 2
  20. 2026WorldFlow3D: Flowing Through 3D Distributions for Unbounded World Generationcited by 2
  21. 2026Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Datacited by 2
  22. 2026HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agentscited by 2
  23. 2025PrITTI: Primitive-based Generation of Controllable and Editable 3D Semantic Urban Scenescited by 1
  24. 2026ScenDi: 3D-to-2D Scene Diffusion Cascades for Urban Generationcited by 1
  25. 2026NuiWorld: Exploring a Scalable Framework for End-to-End Controllable World Generationcited by 1
  26. 2026RoamScene3D: Immersive Text-to-3D Scene Generation via Adaptive Object-aware Roamingcited by 1
  27. 2026Crafting Your Evolving Dreams: Concept-Incremental Versatile Customizationcited by 1
  28. 2026Global-Local Monte Carlo Tree Search in Vision-Language Models for Text-to-3D Indoor Scene Generationcited by 1
  29. 2025ProcGen3D: Learning Neural Procedural Graph Representations for Image-to-3D Reconstructioncited by 0
  30. 2025CC-FMO: Camera-Conditioned Zero-Shot Single Image to 3D Scene Generation with Foundation Model Orchestrationcited by 0
  31. 2026Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flowcited by 0
  32. 2026GardenDesigner: Encoding Aesthetic Principles into Jiangnan Garden Construction via a Chain of Agentscited by 0
  33. 2026Map2World: Segment Map Conditioned Text to 3D World Generationcited by 0
  34. 2026VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangementcited by 0
  35. 2026EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Modelscited by 0
  36. 2026Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantificationcited by 0
  37. 20263D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environmentscited by 0
  38. 2026Hierarchical Flow Matching for 3D Point Cloud Generationcited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
  2. 2024Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation
  3. 2024L3DG: Latent 3D Gaussian Diffusion
  4. 2026Any 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale
  5. 2025Bolt3D: Generating 3D Scenes in Seconds
  6. 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction
  7. 2025From One to More: Contextual Part Latents for 3D Generation
  8. 2024V3D: Video Diffusion Models are Effective 3D Generators