# TerraFusion: Joint Generation of Terrain Geometry and Texture Using Latent Diffusion Models

arXiv: https://arxiv.org/abs/2505.04050 · HTML: https://mlsanity.com/paper/2505.04050/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 13 of its 30 references that are in the map.
- First posted on arXiv 7 May 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [The Role of ImageNet Classes in Fr'echet Inception Distance](https://mlsanity.com/paper/2203.06026/) (2022)
- [Diffusion Model-Based Image Editing: A Survey](https://mlsanity.com/paper/2402.17525/) (2024)
- [PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer](https://mlsanity.com/paper/1909.06956/) (2019)
- [Learning Texture Manifolds with the Periodic Spatial GAN](https://mlsanity.com/paper/1705.06566/) (2017)
- [Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Models](https://mlsanity.com/paper/2503.18446/) (2025)
- [MESA: Text-Driven Terrain Generation Using Latent Diffusion and Global Copernicus Data](https://mlsanity.com/paper/2504.07210/) (2025)
- [Seamless Satellite-image Synthesis](https://mlsanity.com/paper/2111.03384/) (2021)

## Built on by (most cited first, 1 of 1)

- [WorldWeave: Growing Persistent Geometric Worlds for Video Generation](https://mlsanity.com/paper/2609.34221/) (2026)

## Abstract (arXiv)

3D terrain models are essential in fields such as video game development and film production. Since surface color often correlates with terrain geometry, capturing this relationship is crucial to achieving realism. However, most existing methods generate either a heightmap or a texture, without sufficiently accounting for the inherent correlation. In this paper, we propose a method that jointly generates terrain heightmaps and textures using a latent diffusion model. First, we train the model in an unsupervised manner to randomly generate paired heightmaps and textures. Then, we perform supervised learning of an external adapter to enable user control via hand-drawn sketches. Experiments show that our approach allows intuitive terrain generation while preserving the correlation between heightmaps and textures.