mlsanityData to 14 Oct 2026

LayoutPrompter: Awaken the Design Ability of Large Language Models

arXiv 2311.06495 ↗First posted 11 Nov 2023cs.CVcited by 48

Not on the map. Why?

Abstract (arXiv)

Conditional graphic layout generation, which automatically maps user constraints to high-quality layouts, has attracted widespread attention today. Although recent works have achieved promising performance, the lack of versatility and data efficiency hinders their practical applications. In this work, we propose LayoutPrompter, which leverages large language models (LLMs) to address the above problems through in-context learning. LayoutPrompter is made up of three key components, namely input-output serialization, dynamic exemplar selection and layout ranking. Specifically, the input-output serialization component meticulously designs the input and output formats for each layout generation task. Dynamic exemplar selection is responsible for selecting the most helpful prompting exemplars for a given input. And a layout ranker is used to pick the highest quality layout from multiple outputs of LLMs. We conduct experiments on all existing layout generation tasks using four public datasets. Despite the simplicity of our approach, experimental results show that LayoutPrompter can compete with or even outperform state-of-the-art approaches on these tasks without any model training or fine-tuning. This demonstrates the effectiveness of this versatile and training-free approach. In addition, the ablation studies show that LayoutPrompter is significantly superior to the training-based baseline in a low-data regime, further indicating the data efficiency of LayoutPrompter. Our project is available at https://github.com/microsoft/LayoutGeneration/tree/main/LayoutPrompter.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 11 Nov 2023.

Builds on

33 of 40 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2022Emergent Abilities of Large Language Modelscited by 2,512
  10. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  12. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
21 more
  1. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  2. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  3. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  4. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  5. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  6. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
  7. 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 123
  8. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  9. 2021Variational Transformer Networks for Layout Generationcited by 88
  10. 2019Neural Design Network: Graphic Layout Generation with Constraintscited by 83
  11. 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
  12. 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
  13. 2020Attribute-conditioned Layout GAN for Automatic Graphic Designcited by 68
  14. 2023PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layoutcited by 61
  15. 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
  16. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  17. 2023Unifying Layout Generation with a Decoupled Diffusion Modelcited by 41
  18. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
  19. 2023A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptionscited by 15
  20. 2020Auto Completion of User Interface Layout Design Using Transformer-Based Tree Decoderscited by 8
  21. 2021Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Modelscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 23; 2026: 13.

48 papers.

Sort:

By area:

  1. 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
  2. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  3. 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 35
  4. 2024LayoutFlow: Flow Matching for Layout Generationcited by 21
  5. 2024AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generationcited by 20
  6. 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmarkcited by 17
  7. 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16
  8. 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 16
  9. 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 15
  10. 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posterscited by 14
  11. 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Compositioncited by 13
  12. 2025BannerAgency: Advertising Banner Design with Multimodal LLM Agentscited by 11
36 more
  1. 2024LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generationcited by 10
  2. 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Modelscited by 9
  3. 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedbackcited by 9
  4. 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesiscited by 8
  5. 2024Type-R: Automatically Retouching Typos for Text-to-Image Generationcited by 7
  6. 2025LayoutCoT: Unleashing the Deep Reasoning Potential of Large Language Models for Layout Generationcited by 7
  7. 2025SEGA: A Stepwise Evolution Paradigm for Content-Aware Layout Generation with Design Priorcited by 6
  8. 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Designcited by 6
  9. 2026DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generationcited by 5
  10. 2026Any2Poster: Any-Source Poster Generation Across Modalities and Domainscited by 5
  11. 2025SciPostGen: Bridging the Gap between Scientific Papers and Poster Layoutscited by 4
  12. 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMscited by 4
  13. 2026Seeing is Improving: Visual Feedback for Iterative Text Layout Refinementcited by 4
  14. 2024CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusioncited by 3
  15. 2024GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layoutscited by 3
  16. 2025From Fragment to One Piece: A Survey on AI-Driven Graphic Designcited by 2
  17. 2025Relation-Rich Visual Document Generator for Visual Information Extractioncited by 2
  18. 2025Agentic Design Review Systemcited by 2
  19. 2025OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learningcited by 2
  20. 2025Vector Prism: Animating Vector Graphics by Stratifying Semantic Structurecited by 2
  21. 2026PSDesigner: Automated Graphic Design with a Human-Like Creative Workflowcited by 2
  22. 2025SceneVTG++: Controllable Multilingual Visual Text Generation in the Wildcited by 1
  23. 2025LLMs as Layout Designers: Enhanced Spatial Reasoning for Content-Aware Layout Generationcited by 1
  24. 2025DocReward: A Document Reward Model for Structuring and Stylizingcited by 1
  25. 2026AnyDoc: Enhancing Document Generation via Large-Scale HTML/CSS Data Synthesis and Height-Aware Reinforcement Optimizationcited by 1
  26. 2026MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revisioncited by 1
  27. 2026PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slidescited by 1
  28. 2025Generating Animated Layouts as Structured Text Representationscited by 0
  29. 2025Automatic Text Box Placement for Supporting Typographic Designcited by 0
  30. 2025Sketch-to-Layout: Sketch-Guided Multimodal Layout Generationcited by 0
  31. 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generationcited by 0
  32. 2025Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Modelscited by 0
  33. 2026PRISM: Learning Design Knowledge from Data for Stylistic Design Improvementcited by 0
  34. 2026iPoster: Content-Aware Layout Generation for Interactive Poster Design via Graph-Enhanced Diffusion Modelscited by 0
  35. 2026Towards Design Compositingcited by 0
  36. 2026Symbol and Footprint Database for Electronic Components by Agentic Recognition and Generationcited by 0