mlsanityData to 7 Oct 2026

Tell2Design: A Dataset for Language-Guided Floor Plan Generation

arXiv 2311.15941 ↗First posted 27 Nov 2023cs.CL cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

We consider the task of generating designs directly from natural language descriptions, and consider floor plan generation as the initial research area. Language conditional generative models have recently been very successful in generating high-quality artistic images. However, designs must satisfy different constraints that are not present in generating artistic images, particularly spatial and relational constraints. We make multiple contributions to initiate research on this task. First, we introduce a novel dataset, textitTell2Design (T2D), which contains more than 80k floor plan designs associated with natural language instructions. Second, we propose a Sequence-to-Sequence model that can serve as a strong baseline for future research. Third, we benchmark this task with several text-conditional image generation models. We conclude by conducting human evaluations on the generated samples and providing an analysis of human performance. We hope our contributions will propel the research on language-guided design generation forward.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

35 of 61 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  11. 2017Neural Discrete Representation Learningcited by 4,650
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
23 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  6. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  7. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  8. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  9. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  10. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  11. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  12. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  13. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  14. 2019DocRED: A Large-Scale Document-Level Relation Extraction Datasetcited by 239
  15. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  16. 2015Generating Images from Captions with Attentioncited by 135
  17. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  18. 2021HairCLIP: Design Your Hair by Text and Reference Imagecited by 79
  19. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62
  20. 2022Geometry Aligned Variational Transformer for Image-conditioned Layout Generationcited by 33
  21. 2022ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Designcited by 16
  22. 2022DALLE-URBAN: Capturing the urban design expertise of large text to image transformerscited by 12
  23. 2020Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Onlycited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 8; 2026: 8.

19 papers.

Sort:

By area:

  1. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  2. 2025FlairGPT: Repurposing LLMs for Interior Designscited by 22
  3. 2024Generating 3D House Wireframes with Semanticscited by 8
  4. 2026MANSION: Multi-floor lANguage-to-3D Scene generatIOn for loNg-horizon taskscited by 8
  5. 2024Constrained Layout Generation with Factor Graphscited by 6
  6. 2024HouseTune: Two-Stage Floorplan Generation with LLM Assistancecited by 4
  7. 2025FloorPlan-DeepSeek (FPDS): A multimodal approach to floorplan generation using vector-based next room predictioncited by 3
  8. 2025MCP4IFC: IFC-Based Building Design Using Large Language Modelscited by 2
  9. 2025Co-Layout: LLM-driven Co-optimization for Interior Layoutcited by 1
  10. 2026World Craft: Agentic Framework to Create Visualizable Worlds via Textcited by 1
  11. 2026Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Planscited by 1
  12. 2026PlanCraft: Sketch, Refine, and Furnish for Architect-Inspired Progressive 3D Residential Scene Generationcited by 1
7 more
  1. 2025UrbanSense:A Framework for Quantitative Analysis of Urban Streetscapes leveraging Vision Large Language Modelscited by 0
  2. 2025IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environmentcited by 0
  3. 2025GreenPlanner: Practical Floorplan Layout Generation via an Energy-Aware and Function-Feasible Generative Frameworkcited by 0
  4. 2026Unified Vector Floorplan Generation via Markup Representationcited by 0
  5. 2026Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewardscited by 0
  6. 2026HypergraphFormer: Learning Hypergraphs from LLMs for Editable Floor Plan Generationcited by 0
  7. 2026GRE-Diff: Gaussian Room Embeddings for Structured Layout Diffusioncited by 0