LayoutPrompter: Awaken the Design Ability of Large Language Models
Not on the map. Why?
Abstract (arXiv)
Conditional graphic layout generation, which automatically maps user constraints to high-quality layouts, has attracted widespread attention today. Although recent works have achieved promising performance, the lack of versatility and data efficiency hinders their practical applications. In this work, we propose LayoutPrompter, which leverages large language models (LLMs) to address the above problems through in-context learning. LayoutPrompter is made up of three key components, namely input-output serialization, dynamic exemplar selection and layout ranking. Specifically, the input-output serialization component meticulously designs the input and output formats for each layout generation task. Dynamic exemplar selection is responsible for selecting the most helpful prompting exemplars for a given input. And a layout ranker is used to pick the highest quality layout from multiple outputs of LLMs. We conduct experiments on all existing layout generation tasks using four public datasets. Despite the simplicity of our approach, experimental results show that LayoutPrompter can compete with or even outperform state-of-the-art approaches on these tasks without any model training or fine-tuning. This demonstrates the effectiveness of this versatile and training-free approach. In addition, the ablation studies show that LayoutPrompter is significantly superior to the training-based baseline in a low-data regime, further indicating the data efficiency of LayoutPrompter. Our project is available at https://github.com/microsoft/LayoutGeneration/tree/main/LayoutPrompter.
Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 11 Nov 2023.
Builds on
33 of 40 references.
References · 33
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2023Visual Instruction Tuningcited by 7,869
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
- 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
- 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 766
21 more
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
- 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
- 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
- 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
- 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
- 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
- 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 123
- 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
- 2021Variational Transformer Networks for Layout Generationcited by 88
- 2019Neural Design Network: Graphic Layout Generation with Constraintscited by 83
- 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
- 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
- 2020Attribute-conditioned Layout GAN for Automatic Graphic Designcited by 68
- 2023PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layoutcited by 61
- 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
- 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
- 2023Unifying Layout Generation with a Decoupled Diffusion Modelcited by 41
- 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
- 2023A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptionscited by 15
- 2020Auto Completion of User Interface Layout Design Using Transformer-Based Tree Decoderscited by 8
- 2021Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Modelscited by 8
Built on by
Citing papers per year: 2023: 1; 2024: 11; 2025: 23; 2026: 13.
48 papers.
Sort:
By area:
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
- 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 35
- 2024LayoutFlow: Flow Matching for Layout Generationcited by 21
- 2024AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generationcited by 20
- 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmarkcited by 17
- 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16
- 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 16
- 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 15
- 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posterscited by 14
- 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Compositioncited by 13
- 2025BannerAgency: Advertising Banner Design with Multimodal LLM Agentscited by 11
36 more
- 2024LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generationcited by 10
- 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Modelscited by 9
- 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedbackcited by 9
- 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesiscited by 8
- 2024Type-R: Automatically Retouching Typos for Text-to-Image Generationcited by 7
- 2025LayoutCoT: Unleashing the Deep Reasoning Potential of Large Language Models for Layout Generationcited by 7
- 2025SEGA: A Stepwise Evolution Paradigm for Content-Aware Layout Generation with Design Priorcited by 6
- 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Designcited by 6
- 2026DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generationcited by 5
- 2026Any2Poster: Any-Source Poster Generation Across Modalities and Domainscited by 5
- 2025SciPostGen: Bridging the Gap between Scientific Papers and Poster Layoutscited by 4
- 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMscited by 4
- 2026Seeing is Improving: Visual Feedback for Iterative Text Layout Refinementcited by 4
- 2024CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusioncited by 3
- 2024GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layoutscited by 3
- 2025From Fragment to One Piece: A Survey on AI-Driven Graphic Designcited by 2
- 2025Relation-Rich Visual Document Generator for Visual Information Extractioncited by 2
- 2025Agentic Design Review Systemcited by 2
- 2025OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learningcited by 2
- 2025Vector Prism: Animating Vector Graphics by Stratifying Semantic Structurecited by 2
- 2026PSDesigner: Automated Graphic Design with a Human-Like Creative Workflowcited by 2
- 2025SceneVTG++: Controllable Multilingual Visual Text Generation in the Wildcited by 1
- 2025LLMs as Layout Designers: Enhanced Spatial Reasoning for Content-Aware Layout Generationcited by 1
- 2025DocReward: A Document Reward Model for Structuring and Stylizingcited by 1
- 2026AnyDoc: Enhancing Document Generation via Large-Scale HTML/CSS Data Synthesis and Height-Aware Reinforcement Optimizationcited by 1
- 2026MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revisioncited by 1
- 2026PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slidescited by 1
- 2025Generating Animated Layouts as Structured Text Representationscited by 0
- 2025Automatic Text Box Placement for Supporting Typographic Designcited by 0
- 2025Sketch-to-Layout: Sketch-Guided Multimodal Layout Generationcited by 0
- 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generationcited by 0
- 2025Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Modelscited by 0
- 2026PRISM: Learning Design Knowledge from Data for Stylistic Design Improvementcited by 0
- 2026iPoster: Content-Aware Layout Generation for Interactive Poster Design via Graph-Enhanced Diffusion Modelscited by 0
- 2026Towards Design Compositingcited by 0
- 2026Symbol and Footprint Database for Electronic Components by Agentic Recognition and Generationcited by 0
- 2026PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides31 Aug 2026
- 2026Symbol and Footprint Database for Electronic Components by Agentic Recognition and Generation22 Jul 2026
- 2026MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revision15 Jun 2026
- 2026Any2Poster: Any-Source Poster Generation Across Modalities and Domains1 Jun 2026
- 2026Towards Design Compositing16 Apr 2026
- 2026iPoster: Content-Aware Layout Generation for Interactive Poster Design via Graph-Enhanced Diffusion Models31 Mar 2026
- 2026PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow26 Mar 2026
- 2026AnyDoc: Enhancing Document Generation via Large-Scale HTML/CSS Data Synthesis and Height-Aware Reinforcement Optimization26 Mar 2026
- 2026Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement23 Mar 2026
- 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback12 Feb 2026
- 2026DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generation6 Feb 2026
- 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMs21 Jan 2026
36 more
- 2026PRISM: Learning Design Knowledge from Data for Stylistic Design Improvement16 Jan 2026
- 2025Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure16 Dec 2025
- 2025Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Models14 Dec 2025
- 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation9 Dec 2025
- 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design3 Dec 2025
- 2025SciPostGen: Bridging the Gap between Scientific Papers and Poster Layouts27 Nov 2025
- 2025Sketch-to-Layout: Sketch-Guided Multimodal Layout Generation31 Oct 2025
- 2025OmniDocLayout: Towards Diverse Document Layout Generation via Coarse-to-Fine LLM Learning30 Oct 2025
- 2025SEGA: A Stepwise Evolution Paradigm for Content-Aware Layout Generation with Design Prior17 Oct 2025
- 2025DocReward: A Document Reward Model for Structuring and Stylizing13 Oct 2025
- 2025Automatic Text Box Placement for Supporting Typographic Design9 Oct 2025
- 2025LLMs as Layout Designers: Enhanced Spatial Reasoning for Content-Aware Layout Generation21 Sep 2025
- 2025Agentic Design Review System14 Aug 2025
- 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis1 Jul 2025
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design25 May 2025
- 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark21 May 2025
- 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generation6 May 2025
- 2025Generating Animated Layouts as Structured Text Representations2 May 2025
- 2025LayoutCoT: Unleashing the Deep Reasoning Potential of Large Language Models for Layout Generation15 Apr 2025
- 2025Relation-Rich Visual Document Generator for Visual Information Extraction14 Apr 2025
- 2025From Fragment to One Piece: A Survey on AI-Driven Graphic Design24 Mar 2025
- 2025BannerAgency: Advertising Banner Design with Multimodal LLM Agents14 Mar 2025
- 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models1 Mar 2025
- 2025SceneVTG++: Controllable Multilingual Visual Text Generation in the Wild6 Jan 2025
- 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Composition27 Dec 2024
- 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correction5 Dec 2024
- 2024Type-R: Automatically Retouching Typos for Text-to-Image Generation27 Nov 2024
- 2024GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts18 Nov 2024
- 2024Multimodal Markup Document Models for Graphic Design Completion27 Sep 2024
- 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posters29 Jul 2024
- 2024LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generation17 Jul 2024
- 2024AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generation3 Jun 2024
- 2024CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusion18 May 2024
- 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generation1 Apr 2024
- 2024LayoutFlow: Flow Matching for Layout Generation27 Mar 2024
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering28 Nov 2023
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 140
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
- 2024LayoutFlow: Flow Matching for Layout Generationcited by 21
- 2024CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusioncited by 3
- 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generationcited by 0
- 2026iPoster: Content-Aware Layout Generation for Interactive Poster Design via Graph-Enhanced Diffusion Modelscited by 0
- 2026iPoster: Content-Aware Layout Generation for Interactive Poster Design via Graph-Enhanced Diffusion Models31 Mar 2026
- 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generation9 Dec 2025
- 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Design25 May 2025
- 2024CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusion18 May 2024
- 2024LayoutFlow: Flow Matching for Layout Generation27 Mar 2024
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering28 Nov 2023
- 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Modelscited by 9
- 2026Seeing is Improving: Visual Feedback for Iterative Text Layout Refinementcited by 4
- 2024GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layoutscited by 3
- 2025Relation-Rich Visual Document Generator for Visual Information Extractioncited by 2
- 2025Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Modelscited by 0
- 2026Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement23 Mar 2026
- 2025Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Models14 Dec 2025
- 2025Relation-Rich Visual Document Generator for Visual Information Extraction14 Apr 2025
- 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models1 Mar 2025
- 2024GLDesigner: Leveraging Multi-Modal LLMs as Designer for Enhanced Aesthetic Text Glyph Layouts18 Nov 2024
- 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 35
- 2024AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generationcited by 20
- 2025P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmarkcited by 17
- 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16
- 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 16
- 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 15
- 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posterscited by 14
- 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Compositioncited by 13
- 2025BannerAgency: Advertising Banner Design with Multimodal LLM Agentscited by 11
- 2024LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generationcited by 10
- 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedbackcited by 9
- 2025UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesiscited by 8
12 most cited of 31 All 31 (JSON)
- 2026Symbol and Footprint Database for Electronic Components by Agentic Recognition and Generation22 Jul 2026
- 2026MemSlides: A Hierarchical Memory Driven Agent Framework for Personalized Slide Generation with Multi-turn Local Revision15 Jun 2026
- 2026Any2Poster: Any-Source Poster Generation Across Modalities and Domains1 Jun 2026
- 2026Towards Design Compositing16 Apr 2026
- 2026PSDesigner: Automated Graphic Design with a Human-Like Creative Workflow26 Mar 2026
- 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback12 Feb 2026
- 2026DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generation6 Feb 2026
- 2026DARA: Few-shot Budget Allocation in Online Advertising via In-Context Decision Making with RL-Finetuned LLMs21 Jan 2026
- 2026PRISM: Learning Design Knowledge from Data for Stylistic Design Improvement16 Jan 2026
- 2025Vector Prism: Animating Vector Graphics by Stratifying Semantic Structure16 Dec 2025
- 2025PosterCopilot: Toward Layout Reasoning and Controllable Editing for Professional Graphic Design3 Dec 2025
- 2025SciPostGen: Bridging the Gap between Scientific Papers and Poster Layouts27 Nov 2025
12 newest of 31 All 31 (JSON)