# DALLE-URBAN: Capturing the urban design expertise of large text to image transformers

arXiv: https://arxiv.org/abs/2208.04139 · HTML: https://mlsanity.com/paper/2208.04139/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 3 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [Generating Images with Sparse Representations](https://mlsanity.com/paper/2103.03841/) (2021)
- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021)
- [A Note on the Inception Score](https://mlsanity.com/paper/1801.01973/) (2018)
- [Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data](https://mlsanity.com/paper/2103.16607/) (2021)
- [Effectively Unbiased FID and Inception Score and where to find them](https://mlsanity.com/paper/1911.07023/) (2019)
- [Pros and Cons of GAN Evaluation Measures: New Developments](https://mlsanity.com/paper/2103.09396/) (2021)
- [HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models](https://mlsanity.com/paper/1904.01121/) (2019)
- [Discovering the Hidden Vocabulary of DALLE-2](https://mlsanity.com/paper/2206.00169/) (2022)
- [An Improved Evaluation Framework for Generative Adversarial Networks](https://mlsanity.com/paper/1803.07474/) (2018)
- [On Training Sample Memorization: Lessons from Benchmarking Generative Modeling with a Large-scale Competition](https://mlsanity.com/paper/2106.03062/) (2021)
- [Use of Neural Signals to Evaluate the Quality of Generative Adversarial Network Performance in Facial Image Generation](https://mlsanity.com/paper/1811.04172/) (2018)
- [Self-Supervision, Remote Sensing and Abstraction: Representation Learning Across 3 Million Locations](https://mlsanity.com/paper/2203.04445/) (2022)
- [Streetscape augmentation using generative adversarial networks: insights related to health and wellbeing](https://mlsanity.com/paper/1905.06464/) (2019)

## Built on by (most cited first, 12 of 12)

- [Inspecting the Geographical Representativeness of Images from Text-to-Image Models](https://mlsanity.com/paper/2305.11080/) (2023)
- [Tell2Design: A Dataset for Language-Guided Floor Plan Generation](https://mlsanity.com/paper/2311.15941/) (2023)
- [Using Text-to-Image Generation for Architectural Design Ideation](https://mlsanity.com/paper/2304.10182/) (2023)
- [Deepfakes on Demand: the rise of accessible non-consensual deepfake image generators](https://mlsanity.com/paper/2505.03859/) (2025)
- [AI Art in Architecture](https://mlsanity.com/paper/2212.09399/) (2022)
- [Generative AI Models for Different Steps in Architectural Design: A Literature Review](https://mlsanity.com/paper/2404.01335/) (2024)
- [Transformation vs Tradition: Artificial General Intelligence (AGI) for Arts and Humanities](https://mlsanity.com/paper/2310.19626/) (2023)
- [Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information](https://mlsanity.com/paper/2305.01788/) (2023)
- [Sketch-to-Architecture: Generative AI-aided Architectural Design](https://mlsanity.com/paper/2403.20186/) (2024)
- [Understanding Place Identity with Generative AI](https://mlsanity.com/paper/2306.04662/) (2023)
- [Generative AI and the History of Architecture](https://mlsanity.com/paper/2312.15106/) (2023)
- [Hallucination Behavior in Multimodal LLMs Across Agricultural Image Interpretation and Generation Tasks](https://mlsanity.com/paper/2605.27595/) (2026)

## Abstract (arXiv)

Automatically converting text descriptions into images using transformer architectures has recently received considerable attention. Such advances have implications for many applied design disciplines across fashion, art, architecture, urban planning, landscape design and the future tools available to such disciplines. However, a detailed analysis capturing the capabilities of such models, specifically with a focus on the built environment, has not been performed to date. In this work, we investigate the capabilities and biases of such text-to-image methods as it applies to the built environment in detail. We use a systematic grammar to generate queries related to the built environment and evaluate resulting generated images. We generate 1020 different images and find that text to image transformers are robust at generating realistic images across different domains for this use-case. Generated imagery can be found at the github: https://github.com/sachith500/DALLEURBAN