mlsanityData to 14 Oct 2026

Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation

arXiv 2311.13602 ↗First posted 22 Nov 2023cs.CVcited by 35

On the map: Retrieval-augmented generation → Multimodal RAG

Abstract (arXiv)

Content-aware graphic layout generation aims to automatically arrange visual elements along with a given content, such as an e-commerce product image. In this paper, we argue that the current layout generation approaches suffer from the limited training data for the high-dimensional layout structure. We show that a simple retrieval augmentation can significantly improve the generation quality. Our model, which is named Retrieval-Augmented Layout Transformer (RALF), retrieves nearest neighbor layout examples based on an input image and feeds these results into an autoregressive generator. Our model can apply retrieval augmentation to various controllable generation tasks and yield high-quality layouts within a unified architecture. Our extensive experiments show that RALF successfully generates content-aware layouts in both constrained and unconstrained settings and significantly outperforms the baselines.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Multimodal RAG, one of 15 directions in Retrieval-augmented generation. Builds on 39 of its 55 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

39 of 55 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  10. 2017Billion-scale similarity search with GPUscited by 2,497
  11. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  12. 2017Decoupled Weight Decay Regularizationcited by 1,831
27 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  2. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 738
  3. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 434
  4. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  5. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
  6. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 143
  7. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 137
  8. 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 131
  9. 2020LayoutTransformer: Layout Generation and Completion with Self-attentioncited by 123
  10. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  11. 2022Highly Accurate Dichotomous Image Segmentationcited by 110
  12. 2021Variational Transformer Networks for Layout Generationcited by 88
  13. 2019Neural Design Network: Graphic Layout Generation with Constraintscited by 83
  14. 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
  15. 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
  16. 2020Attribute-conditioned Layout GAN for Automatic Graphic Designcited by 68
  17. 2023PosterLayout: A New Benchmark and Approach for Content-aware Visual-Textual Presentation Layoutcited by 61
  18. 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
  19. 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
  20. 2022LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restrictioncited by 43
  21. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
  22. 2023Relation-Aware Diffusion Model for Controllable Poster Layout Generationcited by 37
  23. 2019Progressive Feature Polishing Network for Salient Object Detectioncited by 34
  24. 2022Geometry Aligned Variational Transformer for Image-conditioned Layout Generationcited by 33
  25. 2023DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformercited by 20
  26. 2020Retrieve-Then-Adapt: Example-based Automatic Generation for Proportion-related Infographicscited by 14
  27. 2023Unsupervised Domain Adaption with Pixel-level Discriminator for Image-aware Layout Generationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 19; 2026: 8.

35 papers.

Sort:

By area:

  1. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 60
  2. 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 35
  3. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  4. 2025CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generationcited by 19
  5. 2024Multimodal Markup Document Models for Graphic Design Completioncited by 16
  6. 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 16
  7. 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 15
  8. 2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluationcited by 14
  9. 2024From Elements to Design: A Layered Approach for Automatic Graphic Design Compositioncited by 13
  10. 2024R2GenCSR: Mining Contextual and Residual Information for LLMs-based Radiology Report Generationcited by 12
  11. 2025Rethinking Layered Graphic Design Generation with a Top-Down Approachcited by 10
  12. 2026PosterVerse: A Full-Workflow Framework for Commercial-Grade Poster Generation with HTML-Based Scalable Typographycited by 10
23 more
  1. 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 9
  2. 2025MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generationcited by 9
  3. 2025ProCrop: Learning Aesthetic Image Cropping from Professional Compositionscited by 8
  4. 2025LayoutCoT: Unleashing the Deep Reasoning Potential of Large Language Models for Layout Generationcited by 7
  5. 2025CAL-RAG: Retrieval-Augmented Multi-Agent Generation for Content-Aware Layout Designcited by 7
  6. 2025SEGA: A Stepwise Evolution Paradigm for Content-Aware Layout Generation with Design Priorcited by 6
  7. 2024Design-o-meter: Towards Evaluating and Refining Graphic Designscited by 5
  8. 2025Smaller But Better: Unifying Layout Generation with Smaller Large Language Modelscited by 5
  9. 2025SciPostGen: Bridging the Gap between Scientific Papers and Poster Layoutscited by 4
  10. 2025LayoutRAG: Retrieval-Augmented Model for Content-agnostic Conditional Layout Generationcited by 3
  11. 2025Scan-and-Print: Patch-level Data Summarization and Augmentation for Content-aware Layout Generation in Poster Designcited by 2
  12. 2026PSDesigner: Automated Graphic Design with a Human-Like Creative Workflowcited by 2
  13. 2024LayoutDiT: Exploring Content-Graphic Balance in Layout Generation with Diffusion Transformercited by 1
  14. 2025DesignLab: Designing Slides Through Iterative Detection and Correctioncited by 1
  15. 2025LLMs as Layout Designers: Enhanced Spatial Reasoning for Content-Aware Layout Generationcited by 1
  16. 2025Automatic Text Box Placement for Supporting Typographic Designcited by 0
  17. 2025UniLayDiff: A Unified Diffusion Transformer for Content-Aware Layout Generationcited by 0
  18. 2026PRISM: Learning Design Knowledge from Data for Stylistic Design Improvementcited by 0
  19. 2026CPT: Controllable and Editable Design Variations with Language Modelscited by 0
  20. 2026GAN-based Domain Adaptation for Image-aware Layout Generation in Advertising Poster Designcited by 0
  21. 2026Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designscited by 0
  22. 2026Learning Interaction between Image and Layout Priors for Joint Image-Layout Generation in Design Templatescited by 0
  23. 2026PLAYGROUND: Progressive Layout Generation with Render-Grounded Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models
  2. 2025SlideCoder: Layout-aware RAG-enhanced Hierarchical Slide Generation from Design
  3. 2023Unifying Layout Generation with a Decoupled Diffusion Model
  4. 2025Content-Aware Ad Banner Layout Generation with Two-Stage Chain-of-Thought in Vision Language Models
  5. 2024PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLM
  6. 2024LayoutFlow: Flow Matching for Layout Generation
  7. 2025InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attention
  8. 2025ReLayout: Integrating Relation Reasoning for Content-aware Layout Generation with Multi-modal Large Language Models