# Sketch-to-Architecture: Generative AI-aided Architectural Design

arXiv: https://arxiv.org/abs/2403.20186 · HTML: https://mlsanity.com/paper/2403.20186/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 29 Mar 2024.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Blended Latent Diffusion](https://mlsanity.com/paper/2206.02779/) (2022)
- [House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation](https://mlsanity.com/paper/2003.06988/) (2020)
- [LayerDiffusion: Layered Controlled Image Editing with Diffusion Models](https://mlsanity.com/paper/2305.18676/) (2023)
- [DALLE-URBAN: Capturing the urban design expertise of large text to image transformers](https://mlsanity.com/paper/2208.04139/) (2022)
- [Architext: Language-Driven Generative Architecture Design](https://mlsanity.com/paper/2303.07519/) (2023)

## Built on by (most cited first, 4 of 4)

- [ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation](https://mlsanity.com/paper/2405.10508/) (2024)
- [Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Framework](https://mlsanity.com/paper/2408.08054/) (2024)
- [Generating Daylight-driven Architectural Design via Diffusion Models](https://mlsanity.com/paper/2404.13353/) (2024)
- [Challenges and Opportunities in 3D Content Generation](https://mlsanity.com/paper/2405.15335/) (2024)

## Abstract (arXiv)

Recently, the development of large-scale models has paved the way for various interdisciplinary research, including architecture. By using generative AI, we present a novel workflow that utilizes AI models to generate conceptual floorplans and 3D models from simple sketches, enabling rapid ideation and controlled generation of architectural renderings based on textual descriptions. Our work demonstrates the potential of generative AI in the architectural design process, pointing towards a new direction of computer-aided architectural design. Our project website is available at: https://zrealli.github.io/sketch2arc