mlsanityData to 7 Oct 2026

Generative AI Models for Different Steps in Architectural Design: A Literature Review

arXiv 2404.01335 ↗First posted 30 Mar 2024cs.LG cs.AIcited by 12

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recent advances in generative artificial intelligence (AI) technologies have been significantly driven by models such as generative adversarial networks (GANs), variational autoencoders (VAEs), and denoising diffusion probabilistic models (DDPMs). Although architects recognize the potential of generative AI in design, personal barriers often restrict their access to the latest technological developments, thereby causing the application of generative AI in architectural design to lag behind. Therefore, it is essential to comprehend the principles and advancements of generative AI models and analyze their relevance in architecture applications. This paper first provides an overview of generative AI technologies, with a focus on probabilistic diffusion models (DDPMs), 3D generative models, and foundation models, highlighting their recent developments and main application scenarios. Then, the paper explains how the abovementioned models could be utilized in architecture. We subdivide the architectural design process into six steps and review related research projects in each step from 2020 to the present. Lastly, this paper discusses potential future directions for applying generative AI in the architectural design steps. This research can help architects quickly understand the development and latest progress of generative AI and contribute to the further development of intelligent architecture.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 53 of its 184 references that are in the map. First posted on arXiv 30 Mar 2024.

Builds on

53 of 184 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  2. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  3. 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising23 Nov
  4. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
36 more
  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  6. 2022Video Diffusion Modelscited by 2,280
  7. 2014Conditional Generative Adversarial Netscited by 2,152
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  11. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  12. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  13. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  14. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  15. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  16. 2018Image Generation from Scene Graphscited by 324
  17. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
  18. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  19. 2015DRAW: A Recurrent Neural Network For Image Generationcited by 190
  20. 2018Image Generation from Layoutcited by 105
  21. 2020House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generationcited by 86
  22. 2018SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesiscited by 81
  23. 2018Learning Shape Priors for Single-View 3D Completion and Reconstructioncited by 68
  24. 2020Generative Layout Modeling using Constraint Graphscited by 46
  25. 2021Context-Aware Layout to Image Generation with Enhanced Object Appearancecited by 41
  26. 2023SceneGenie: Scene Graph Guided Diffusion Models for Image Synthesiscited by 41
  27. 2021Learning Anchored Unsigned Distance Functions with Gradient Direction Alignment for Single-view Garment Reconstructioncited by 36
  28. 2023Using Text-to-Image Generation for Architectural Design Ideationcited by 16
  29. 2021ActFloor-GAN: Activity-Guided Adversarial Networks for Human-Centric Floorplan Designcited by 12
  30. 2022DALLE-URBAN: Capturing the urban design expertise of large text to image transformerscited by 12
  31. 2023Conditional 360-degree Image Synthesis for Immersive Indoor Scene Decorationcited by 9
  32. 2023AniFaceDrawing: Anime Portrait Exploration during Your Sketchingcited by 7
  33. 2023Pika: Empowering Non-Programmers to Author Executable Governance Policies in Online Communitiescited by 5
  34. 2022Floorplan Restoration by Structure Hallucinating Transformer Cascadescited by 4
  35. 2023City Street Layout Generation via Conditional Adversarial Learningcited by 3
  36. 2021Early-Phase Performance-Driven Design using Generative Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 5.

12 papers.

  1. 2024Text2BIM: Generating Building Models Using a Large Language Model-based Multi-Agent Frameworkcited by 9
  2. 2024Exploring Gen-AI applications in building research and industry: A reviewcited by 2
  3. 2025Text-to-Layout: A Generative Workflow for Drafting Architectural Floor Plans Using LLMscited by 2
  4. 2025Multi-View Depth Consistent Image Generation Using Generative AI Models: Application on Architectural Design of University Buildingscited by 0
  5. 2025From Architectural Sketch to Conceptual Representation: Using Structure-Aware Diffusion Model to Generate Renderings of School Buildingscited by 0
  6. 2025When Discourse Stalls: Moving Past Five Semantic Stopsigns about Generative AI in Design Researchcited by 0
  7. 2025IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environmentcited by 0
  8. 2026Sketch-Based Facade Renovation With Generative AI: A Streamlined Framework for Bypassing As-Built Modelling in Industrial Adaptive Reusecited by 0
  9. 2026Generative AI Assisted Workflows in Architectural Conceptual Design: Performance, Creative Self-Efficacy, and Cognitive Loadcited by 0
  10. 2026LoD Sketch Extraction from Architectural Models Using Generative AI: Dataset Construction for Multi-Level Architectural Design Generationcited by 0
  11. 2026ONOTE: Hypergraph-Grounded Omnimodal Reasoning for Computational Music Sciencecited by 0
  12. 2026Generative AI and Extended Reality in Collaborative Architectural Design Education: An Exploratory Studio Studycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
  2. 2025Generative AI for Urban Design: A Stepwise Approach Integrating Human Expertise with Multimodal Diffusion Models
  3. 2023Automating Computational Design with Generative AI
  4. 2024Generative AI in Vision: A Survey on Models, Metrics and Applications
  5. 2025Generative AI for Vision: A Comprehensive Study of Frameworks and Applications
  6. 2024Generating floorplans for various building functionalities via latent diffusion model
  7. 2023Beyond Statistical Similarity: Rethinking Metrics for Deep Generative Models in Engineering Design
  8. 2024Generating Daylight-driven Architectural Design via Diffusion Models