# Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion

arXiv: https://arxiv.org/abs/2401.10786 · HTML: https://mlsanity.com/paper/2401.10786/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 39 of its 44 references that are in the map.
- First posted on arXiv 19 Jan 2024.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 39)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion](https://mlsanity.com/paper/2307.01097/) (2023)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)
- [HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion](https://mlsanity.com/paper/2303.17015/) (2023)
- [Unconstrained Scene Generation with Locally Conditioned Radiance Fields](https://mlsanity.com/paper/2104.00670/) (2021)
- [GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds](https://mlsanity.com/paper/2104.07659/) (2021)
- [NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Models](https://mlsanity.com/paper/2304.09787/) (2023)

## Built on by (most cited first, 27 of 27)

- [Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives](https://mlsanity.com/paper/2404.08926/) (2024)
- [CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesis](https://mlsanity.com/paper/2408.14765/) (2024)
- [Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery](https://mlsanity.com/paper/2510.15869/) (2025)
- [Sat2City: 3D City Generation from A Single Satellite Image with Cascaded Latent Diffusion](https://mlsanity.com/paper/2507.04403/) (2025)
- [Geospecific View Generation -- Geometry-Context Aware High-resolution Ground View Inference from Satellite Views](https://mlsanity.com/paper/2407.08061/) (2024)
- [Decoupled Diffusion Sparks Adaptive Scene Generation](https://mlsanity.com/paper/2504.10485/) (2025)
- [Seeing through Satellite Images at Street Views](https://mlsanity.com/paper/2505.17001/) (2025)
- [Towards Scalable Spatial Intelligence via 2D-to-3D Data Lifting](https://mlsanity.com/paper/2507.18678/) (2025)
- [Skyeyes: Ground Roaming using Aerial View Images](https://mlsanity.com/paper/2409.16685/) (2024)
- [Sat3DGen: Comprehensive Street-Level 3D Scene Generation from Single Satellite Image](https://mlsanity.com/paper/2605.14984/) (2026)
- [Mixed-View Panorama Synthesis using Geospatially Guided Diffusion](https://mlsanity.com/paper/2407.09672/) (2024)
- [Satellite to GroundScape -- Large-scale Consistent Ground View Generation from Satellite Views](https://mlsanity.com/paper/2504.15786/) (2025)
- [Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image](https://mlsanity.com/paper/2606.24138/) (2026)
- [Automatic Scene Generation: State-of-the-Art Techniques, Models, Datasets, Challenges, and Future Prospects](https://mlsanity.com/paper/2410.01816/) (2024)
- [DiffusionUavLoc: Visually Prompted Diffusion for Cross-View UAV Localization](https://mlsanity.com/paper/2511.06422/) (2025)
- [ABot-Earth 0.5: Generative 3D Earth Model](https://mlsanity.com/paper/2606.09967/) (2026)
- [GS-Voxel: Fitting-Free Structured Latents for Large-Scale 3DGS Generation](https://mlsanity.com/paper/2608.17988/) (2026)
- [AerialGo: Walking-through City View Generation from Aerial Perspectives](https://mlsanity.com/paper/2412.00157/) (2024)
- [Language-Informed Hyperspectral Image Synthesis for Imbalanced-Small Sample Classification via Semi-Supervised Conditional Diffusion Model](https://mlsanity.com/paper/2502.19700/) (2025)
- [ScenDi: 3D-to-2D Scene Diffusion Cascades for Urban Generation](https://mlsanity.com/paper/2601.15221/) (2026)
- [Bird Eye-View to Street-View: A Survey](https://mlsanity.com/paper/2405.08961/) (2024)
- [SatDreamer360: Multiview-Consistent Generation of Ground-Level Scenes from Satellite Imagery](https://mlsanity.com/paper/2506.00600/) (2025)
- [Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Dataset](https://mlsanity.com/paper/2512.10321/) (2025)
- [SAM 3D for 3D Object Reconstruction from Remote Sensing Images](https://mlsanity.com/paper/2512.22452/) (2025)
- [Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images](https://mlsanity.com/paper/2605.19656/) (2026)
- [GeoFidelity-Bench: Evaluating Block-Conditioned Geographic Fidelity in Text-to-Image Street-View Generation](https://mlsanity.com/paper/2606.23669/) (2026)
- [CrossTimeEdit: A Decade-Spanning Cross-View Dataset and Reward-Guided Editing for Historical Street-View Generation](https://mlsanity.com/paper/2609.36616/) (2026)

## Abstract (arXiv)

Directly generating scenes from satellite imagery offers exciting possibilities for integration into applications like games and map services. However, challenges arise from significant view changes and scene scale. Previous efforts mainly focused on image or video generation, lacking exploration into the adaptability of scene generation for arbitrary views. Existing 3D generation works either operate at the object level or are difficult to utilize the geometry obtained from satellite imagery. To overcome these limitations, we propose a novel architecture for direct 3D scene generation by introducing diffusion models into 3D sparse representations and combining them with neural rendering techniques. Specifically, our approach generates texture colors at the point level for a given geometry using a 3D diffusion model first, which is then transformed into a scene representation in a feed-forward manner. The representation can be utilized to render arbitrary views which would excel in both single-frame quality and inter-frame consistency. Experiments in two city-scale datasets show that our model demonstrates proficiency in generating photo-realistic street-view image sequences and cross-view urban scenes from satellite imagery.