# Constrained Layout Generation with Factor Graphs

arXiv: https://arxiv.org/abs/2404.00385 · HTML: https://mlsanity.com/paper/2404.00385/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 30 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Photographic Image Synthesis with Cascaded Refinement Networks](https://mlsanity.com/paper/1707.09405/) (2017)
- [LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators](https://mlsanity.com/paper/1901.06767/) (2019)
- [Variational Transformer Networks for Layout Generation](https://mlsanity.com/paper/2104.02416/) (2021)
- [House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation](https://mlsanity.com/paper/2003.06988/) (2020)
- [Neural Design Network: Graphic Layout Generation with Constraints](https://mlsanity.com/paper/1912.09421/) (2019)
- [Constrained Graphic Layout Generation via Latent Optimization](https://mlsanity.com/paper/2108.00871/) (2021)
- [HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising](https://mlsanity.com/paper/2211.13287/) (2022)
- [Neural Enhanced Belief Propagation on Factor Graphs](https://mlsanity.com/paper/2003.01998/) (2020)
- [Generative Layout Modeling using Constraint Graphs](https://mlsanity.com/paper/2011.13417/) (2020)
- [Inference in Probabilistic Graphical Models by Graph Neural Networks](https://mlsanity.com/paper/1803.07710/) (2018)
- [Geometry Aligned Variational Transformer for Image-conditioned Layout Generation](https://mlsanity.com/paper/2209.00852/) (2022)
- [Building-GAN: Graph-Conditioned Architectural Volumetric Design Generation](https://mlsanity.com/paper/2104.13316/) (2021)
- [Tell2Design: A Dataset for Language-Guided Floor Plan Generation](https://mlsanity.com/paper/2311.15941/) (2023)
- [Factor Graph Neural Networks](https://mlsanity.com/paper/2308.00887/) (2023)
- [Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only](https://mlsanity.com/paper/2003.00397/) (2020)
- [LEO: Learning Energy-based Models in Factor Graph Optimization](https://mlsanity.com/paper/2108.02274/) (2021)
- [iPLAN: Interactive and Procedural Layout Planning](https://mlsanity.com/paper/2203.14412/) (2022)
- [Neuralizing Efficient Higher-order Belief Propagation](https://mlsanity.com/paper/2010.09283/) (2020)

## Built on by (most cited first, 7 of 7)

- [ChatHouseDiffusion: Prompt-Guided Generation and Editing of Floor Plans](https://mlsanity.com/paper/2410.11908/) (2024)
- [Generation of Uncertainty-Aware High-Level Spatial Concepts in Factorized 3D Scene Graphs via Graph Neural Networks](https://mlsanity.com/paper/2409.11972/) (2024)
- [Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans](https://mlsanity.com/paper/2603.11640/) (2026)
- [When Diffusion Breaks Constraints: Sequential Autoregressive Generation with RL and MCTS](https://mlsanity.com/paper/2512.01242/) (2025)
- [GFLAN: Generative Functional Layouts](https://mlsanity.com/paper/2512.16275/) (2025)
- [GRE-Diff: Gaussian Room Embeddings for Structured Layout Diffusion](https://mlsanity.com/paper/2607.08086/) (2026)
- [VIGOR: Verifiable Industrial Geometry Optimization and Repair in Scene Programs](https://mlsanity.com/paper/2610.13145/) (2026)

## Abstract (arXiv)

This paper addresses the challenge of object-centric layout generation under spatial constraints, seen in multiple domains including floorplan design process. The design process typically involves specifying a set of spatial constraints that include object attributes like size and inter-object relations such as relative positioning. Existing works, which typically represent objects as single nodes, lack the granularity to accurately model complex interactions between objects. For instance, often only certain parts of an object, like a room's right wall, interact with adjacent objects. To address this gap, we introduce a factor graph based approach with four latent variable nodes for each room, and a factor node for each constraint. The factor nodes represent dependencies among the variables to which they are connected, effectively capturing constraints that are potentially of a higher order. We then develop message-passing on the bipartite graph, forming a factor graph neural network that is trained to produce a floorplan that aligns with the desired requirements. Our approach is simple and generates layouts faithful to the user requirements, demonstrated by a large improvement in IOU scores over existing methods. Additionally, our approach, being inferential and accurate, is well-suited to the practical human-in-the-loop design process where specifications evolve iteratively, offering a practical and powerful tool for AI-guided design.