# Octree Diffusion for Semantic Scene Generation and Completion

arXiv: https://arxiv.org/abs/2509.16483 · HTML: https://mlsanity.com/paper/2509.16483/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 21 of 21)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2411.15139/) (2024)
- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.12251/) (2023)
- [Diffusion-SDF: Conditional Generative Modeling of Signed Distance Functions](https://mlsanity.com/paper/2211.13757/) (2022)
- [Generative and Discriminative Voxel Modeling with Convolutional Neural Networks](https://mlsanity.com/paper/1608.04236/) (2016)
- [Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations](https://mlsanity.com/paper/2205.02825/) (2022)
- [SemCity: Semantic Scene Generation with Triplane Diffusion](https://mlsanity.com/paper/2403.07773/) (2024)
- [OctFusion: Octree-based Diffusion Models for 3D Shape Generation](https://mlsanity.com/paper/2408.14732/) (2024)
- [Reflecting Reality: Enabling Diffusion Models to Produce Faithful Mirror Reflections](https://mlsanity.com/paper/2409.14677/) (2024)
- [DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2409.18092/) (2024)
- [SceneSense: Diffusion Models for 3D Occupancy Synthesis from Partial Observation](https://mlsanity.com/paper/2403.11985/) (2024)
- [Online Diffusion-Based 3D Occupancy Prediction at the Frontier with Probabilistic Map Reconciliation](https://mlsanity.com/paper/2409.10681/) (2024)
- [CU-Multi: A Dataset for Multi-Robot Data Association](https://mlsanity.com/paper/2505.17576/) (2025)

## Built on by (most cited first, 1 of 1)

- [Generative Semantic Scene Completion](https://mlsanity.com/paper/2608.26737/) (2026)

## Abstract (arXiv)

The completion, extension, and generation of 3D semantic scenes are an interrelated set of capabilities that are useful for robotic navigation and exploration. Existing approaches seek to decouple these problems and solve them one-off. Additionally, these approaches are often domain-specific, requiring separate models for different data distributions, e.g. indoor vs. outdoor scenes. To unify these techniques and provide cross-domain compatibility, we develop a single framework that can perform scene completion, extension, and generation in both indoor and outdoor scenes, which we term Octree Latent Semantic Diffusion. Our approach operates directly on an efficient dual octree graph latent representation: a hierarchical, sparse, and memory-efficient occupancy structure. This technique disentangles synthesis into two stages: (i) structure diffusion, which predicts binary split signals to construct a coarse occupancy octree, and (ii) latent semantic diffusion, which generates semantic embeddings decoded by a graph VAE into voxel-level semantic labels. To perform semantic scene completion or extension, our model leverages inference-time latent inpainting, or outpainting respectively. These inference-time methods use partial LiDAR scans or maps to condition generation, without the need for retraining or finetuning. We demonstrate high-quality structure, coherent semantics, and robust completion from single LiDAR scans, as well as zero-shot generalization to out-of-distribution LiDAR data. These results indicate that completion-through-generation in a dual octree graph latent space is a practical and scalable alternative to regression-based pipelines for real-world robotic perception tasks.