mlsanityData to 7 Oct 2026

Variational Transformer Networks for Layout Generation

arXiv 2104.02416 ↗First posted 6 Apr 2021cs.CV cs.LGcited by 88

Not on the map. Why?

Abstract (arXiv)

Generative models able to synthesize layouts of different kinds (e.g. documents, user interfaces or furniture arrangements) are a useful tool to aid design processes and as a first step in the generation of synthetic data, among other tasks. We exploit the properties of self-attention layers to capture high level relationships between elements in a layout, and use these as the building blocks of the well-known Variational Autoencoder (VAE) formulation. Our proposed Variational Transformer Network (VTN) is capable of learning margins, alignments and other global design rules without explicit supervision. Layouts sampled from our model have a high degree of resemblance to the training data, while demonstrating appealing diversity. In an extensive evaluation on publicly available benchmarks for different layout types VTNs achieve state-of-the-art diversity and perceptual quality. Additionally, we show the capabilities of this method as part of a document layout detection pipeline.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 6 Apr 2021.

Builds on

26 of 43 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  11. 2015Generating Sentences from a Continuous Spacecited by 607
  12. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
14 more
  1. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  2. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  3. 2019PubLayNet: largest dataset ever for document layout analysiscited by 349
  4. 2016Variational Lossy Autoencodercited by 175
  5. 2019LayoutGAN: Generating Graphic Layouts with Wireframe Discriminatorscited by 130
  6. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  7. 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
  8. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  9. 2017Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Networkcited by 92
  10. 2019Image Synthesis From Reconfigurable Layout and Stylecited by 85
  11. 2019Neural Design Network: Graphic Layout Generation with Constraintscited by 83
  12. 2019READ: Recursive Autoencoders for Document Layout Generationcited by 51
  13. 2020Variational Transformers for Diverse Response Generationcited by 36
  14. 2017Automatic Generation of Constrained Furniture Layoutscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 12; 2023: 21; 2024: 25; 2025: 16; 2026: 6.

88 papers.

Sort:

By area:

  1. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  2. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generationcited by 141
  3. 2021CanvasVAE: Learning to Generate Vector Graphic Documentscited by 93
  4. 2021Constrained Graphic Layout Generation via Latent Optimizationcited by 83
  5. 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
  6. 2022Composition-aware Graphic Layout GAN for Visual-textual Presentation Designscited by 59
  7. 2023LayoutDM: Transformer-based Diffusion Model for Layout Generationcited by 57
  8. 2023LayoutPrompter: Awaken the Design Ability of Large Language Modelscited by 47
  9. 2022LayoutFormer++: Conditional Graphic Layout Generation via Constraint Serialization and Decoding Space Restrictioncited by 43
  10. 2023Unifying Layout Generation with a Decoupled Diffusion Modelcited by 41
  11. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 40
  12. 2024PosterLLaVa: Constructing a Unified Multi-modal Layout Generator with LLMcited by 39
38 more
  1. 2024Graphic Design with Large Multimodal Modelcited by 38
  2. 2022OCR-IDL: OCR Annotations for Industry Document Library Datasetcited by 35
  3. 2023Retrieval-Augmented Layout Transformer for Content-Aware Layout Generationcited by 34
  4. 2024PosterLlama: Bridging Design Ability of Langauge Model to Contents-Aware Layout Generationcited by 34
  5. 2022Geometry Aligned Variational Transformer for Image-conditioned Layout Generationcited by 33
  6. 2023Towards Flexible Multi-modal Document Modelscited by 30
  7. 2024Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraintscited by 26
  8. 2023PLay: Parametrically Conditioned Layout Generation using Latent Diffusioncited by 24
  9. 2023Dolfin: Diffusion Layout Transformers without Autoencodercited by 23
  10. 2023Diffusion-based Document Layout Generationcited by 21
  11. 2024LayoutFlow: Flow Matching for Layout Generationcited by 21
  12. 2023DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformercited by 20
  13. 2022Aesthetic Text Logo Synthesis via Content-aware Layout Inferringcited by 19
  14. 2024Desigen: A Pipeline for Controllable Design Template Generationcited by 19
  15. 2023SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Controlcited by 18
  16. 2023LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Modelscited by 17
  17. 2021Go with the Flows: Mixtures of Normalizing Flows for Point Cloud Generation and Reconstructioncited by 16
  18. 2022LayoutDETR: Detection Transformer Is a Good Multimodal Layout Designercited by 15
  19. 2023GlobalMapper: Arbitrary-Shaped Urban Layout Generationcited by 15
  20. 2023A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptionscited by 15
  21. 2023UI Layout Generation with LLMs Guided by UI Grammarcited by 15
  22. 2023Planning and Rendering: Towards Product Poster Generation with Diffusion Modelscited by 15
  23. 2025PosterO: Structuring Layout Trees to Enable Language Models in Generalized Content-Aware Layout Generationcited by 15
  24. 2024SciPostLayout: A Dataset for Layout Analysis and Layout Generation of Scientific Posterscited by 14
  25. 2023Unsupervised Domain Adaption with Pixel-level Discriminator for Image-aware Layout Generationcited by 13
  26. 2025Uni-Layout: Integrating Human Feedback in Unified Layout Generation and Evaluationcited by 13
  27. 2024RoDLA: Benchmarking the Robustness of Document Layout Analysis Modelscited by 12
  28. 2022CapOnImage: Context-driven Dense-Captioning on Imagecited by 11
  29. 2023Towards Diverse and Consistent Typography Generationcited by 11
  30. 2021CRD-CGAN: Category-Consistent and Relativistic Constraints for Diverse Text-to-Image Generationcited by 10
  31. 2022Exploiting Pretrained Biochemical Language Models for Targeted Drug Designcited by 10
  32. 2024COHO: Context-Sensitive City-Scale Hierarchical Urban Layout Generationcited by 10
  33. 2024LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generationcited by 10
  34. 2023GDB: Gated convolutions-based Document Binarizationcited by 9
  35. 2021Creating User Interface Mock-ups from High-Level Text Descriptions with Deep-Learning Modelscited by 8
  36. 2025AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Modelscited by 8
  37. 2025Visual Variational Autoencoder Prompt Tuningcited by 8
  38. 2024TextLap: Customizing Language Models for Text-to-Layout Planningcited by 7

50 most cited of 88 All 88 (JSON)