# Diverse Multimedia Layout Generation with Multi Choice Learning

arXiv: https://arxiv.org/abs/2301.06629 · HTML: https://mlsanity.com/paper/2301.06629/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 16 Jan 2023.

## Builds on (most cited first, 19 of 19)

- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators](https://mlsanity.com/paper/1901.06767/) (2019)
- [LayoutVAE: Stochastic Scene Layout Generation From a Label Set](https://mlsanity.com/paper/1907.10719/) (2019)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [CAN: Creative Adversarial Networks, Generating "Art" by Learning About Styles and Deviating from Style Norms](https://mlsanity.com/paper/1706.07068/) (2017)
- [Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction](https://mlsanity.com/paper/1906.03631/) (2019)
- [Neural Design Network: Graphic Layout Generation with Constraints](https://mlsanity.com/paper/1912.09421/) (2019)
- [Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation](https://mlsanity.com/paper/1806.10729/) (2018)
- [Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles](https://mlsanity.com/paper/1606.07839/) (2016)
- [Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective](https://mlsanity.com/paper/1806.07772/) (2018)
- [READ: Recursive Autoencoders for Document Layout Generation](https://mlsanity.com/paper/1909.00302/) (2019)
- [Deep Directional Statistics: Pose Estimation with Uncertainty Quantification](https://mlsanity.com/paper/1805.03430/) (2018)
- [GRAINS: Generative Recursive Autoencoders for INdoor Scenes](https://mlsanity.com/paper/1807.09193/) (2018)
- [Action Anticipation By Predicting Future Dynamic Images](https://mlsanity.com/paper/1808.00141/) (2018)
- [DiverseNet: When One Right Answer is not Enough](https://mlsanity.com/paper/2008.10634/) (2020)

## Built on by (most cited first, 8 of 8)

- [LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models](https://mlsanity.com/paper/2303.11589/) (2023)
- [PLay: Parametrically Conditioned Layout Generation using Latent Diffusion](https://mlsanity.com/paper/2301.11529/) (2023)
- [Multimodal Markup Document Models for Graphic Design Completion](https://mlsanity.com/paper/2409.19051/) (2024)
- [LTSim: Layout Transportation-based Similarity Measure for Evaluating Layout Generation](https://mlsanity.com/paper/2407.12356/) (2024)
- [Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligence](https://mlsanity.com/paper/2601.12318/) (2026)
- [CoLay: Controllable Layout Generation through Multi-conditional Latent Diffusion](https://mlsanity.com/paper/2405.13045/) (2024)
- [Multiple Hypothesis Dropout: Estimating the Parameters of Multi-Modal Output Distributions](https://mlsanity.com/paper/2312.11735/) (2023)
- [Learning to Kern: Set-wise Estimation of Optimal Letter Space](https://mlsanity.com/paper/2402.14313/) (2024)

## Abstract (arXiv)

Designing visually appealing layouts for multimedia documents containing text, graphs and images requires a form of creative intelligence. Modelling the generation of layouts has recently gained attention due to its importance in aesthetics and communication style. In contrast to standard prediction tasks, there are a range of acceptable layouts which depend on user preferences. For example, a poster designer may prefer logos on the top-left while another prefers logos on the bottom-right. Both are correct choices yet existing machine learning models treat layouts as a single choice prediction problem. In such situations, these models would simply average over all possible choices given the same input forming a degenerate sample. In the above example, this would form an unacceptable layout with a logo in the centre. In this paper, we present an auto-regressive neural network architecture, called LayoutMCL, that uses multi-choice prediction and winner-takes-all loss to effectively stabilise layout generation. LayoutMCL avoids the averaging problem by using multiple predictors to learn a range of possible options for each layout object. This enables LayoutMCL to generate multiple and diverse layouts from a single input which is in contrast with existing approaches which yield similar layouts with minor variations. Through quantitative benchmarks on real data (magazine, document and mobile app layouts), we demonstrate that LayoutMCL reduces Fr'echet Inception Distance (FID) by 83-98% and generates significantly more diversity in comparison to existing approaches.