# LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

arXiv: https://arxiv.org/abs/2407.12025 · HTML: https://mlsanity.com/paper/2407.12025/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 87 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Builds on (most cited first, 19 of 19)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [A Survey on Large Language Model based Autonomous Agents](https://mlsanity.com/paper/2308.11432/) (2023)
- [AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors](https://mlsanity.com/paper/2308.10848/) (2023)
- [BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing](https://mlsanity.com/paper/2305.14720/) (2023)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning](https://mlsanity.com/paper/2307.06135/) (2023)
- [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](https://mlsanity.com/paper/2307.05300/) (2023)
- [RoCo: Dialectic Multi-Robot Collaboration with Large Language Models](https://mlsanity.com/paper/2307.04738/) (2023)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation](https://mlsanity.com/paper/2003.06988/) (2020)
- [Designing Creative AI Partners with COFI: A Framework for Modeling Interaction in Human-AI Co-Creative Systems](https://mlsanity.com/paper/2204.07666/) (2022)
- [A Survey on Large Language Model Hallucination via a Creativity Perspective](https://mlsanity.com/paper/2402.06647/) (2024)
- [The SocialAI School: Insights from Developmental Psychology Towards Artificial Socio-Cultural Agents](https://mlsanity.com/paper/2307.07871/) (2023)
- [Creative Agents: Empowering Agents with Imagination for Creative Tasks](https://mlsanity.com/paper/2312.02519/) (2023)
- [Understanding Nonlinear Collaboration between Human and AI Agents: A Co-design Framework for Creative Design](https://mlsanity.com/paper/2401.07312/) (2024)
- [ActFloor-GAN: Activity-Guided Adversarial Networks for Human-Centric Floorplan Design](https://mlsanity.com/paper/2111.03545/) (2021)
- [Learning Optimal Solutions via an LSTM-Optimization Framework](https://mlsanity.com/paper/2207.02937/) (2022)
- [Learning Personal Style from Few Examples](https://mlsanity.com/paper/2105.14457/) (2021)

## Built on by (most cited first, 1 of 1)

- [Text-to-Layout: A Generative Workflow for Drafting Architectural Floor Plans Using LLMs](https://mlsanity.com/paper/2509.00543/) (2025)

## Abstract (arXiv)

This study introduces LLM4DESIGN, a highly automated system for generating architectural and environmental design proposals. LLM4DESIGN, relying solely on site conditions and design requirements, employs Multi-Agent systems to foster creativity, Retrieval Augmented Generation (RAG) to ground designs in realism, and Visual Language Models (VLM) to synchronize all information. This system resulting in coherent, multi-illustrated, and multi-textual design schemes. The system meets the dual needs of narrative storytelling and objective drawing presentation in generating architectural and environmental design proposals. Extensive comparative and ablation experiments confirm the innovativeness of LLM4DESIGN's narrative and the grounded applicability of its plans, demonstrating its superior performance in the field of urban renewal design. Lastly, we have created the first cross-modal design scheme dataset covering architecture, landscape, interior, and urban design, providing rich resources for future research.