mlsanityData to 7 Oct 2026

TextLap: Customizing Language Models for Text-to-Layout Planning

arXiv 2410.12844 ↗First posted 9 Oct 2024cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Automatic generation of graphical layouts is crucial for many real-world applications, including designing posters, flyers, advertisements, and graphical user interfaces. Given the incredible ability of Large language models (LLMs) in both natural language understanding and generation, we believe that we could customize an LLM to help people create compelling graphical layouts starting with only text instructions from the user. We call our method TextLap (text-based layout planning). It uses a curated instruction-based layout planning dataset (InsLap) to customize LLMs as a graphic designer. We demonstrate the effectiveness of TextLap and show that it outperforms strong baselines, including GPT-4 based methods, for image generation and graphical design benchmarks.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 31 of its 43 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

31 of 43 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  11. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  12. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
19 more
  1. 2023TextDiffuser: Diffusion Models as Text Painterscited by 223
  2. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 141
  3. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 136
  4. 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 122
  5. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  6. 2024InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Priorcited by 110
  7. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 95
  8. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 93
  9. 2021Variational Transformer Networks for Layout Generationcited by 88
  10. 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
  11. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agentscited by 77
  12. 2020Attribute-conditioned Layout GAN for Automatic Graphic Designcited by 67
  13. 2023PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layoutcited by 60
  14. 2022LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restrictioncited by 43
  15. 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraintscited by 26
  16. 2022LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designercited by 15
  17. 2022Scene Graph Expansion for Semantics-Guided Image Outpaintingcited by 11
  18. 2024TRINS: Towards Multimodal Language Models that Can Readcited by 7
  19. 2023Text2Poster: Laying out Stylized Texts on Retrieved Imagescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generationcited by 37
  2. 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 15
  3. 2025Generative AI for Autonomous Driving: Frontiers and Opportunitiescited by 7
  4. 2025MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Modelscited by 4
  5. 2025From Fragment to One Piece: A Survey on AI-Driven Graphic Designcited by 2
  6. 2026Seeing is Improving: Visual Feedback for Iterative Text Layout Refinementcited by 2
  7. 2026Qualitative Evaluation of LLM-Designed GUIcited by 0