# Graph Transformer GANs for Graph-Constrained House Generation

arXiv: https://arxiv.org/abs/2303.08225 · HTML: https://mlsanity.com/paper/2303.08225/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 57 references that are in the map.
- First posted on arXiv 14 Mar 2023.

## Builds on (most cited first, 30 of 48)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [OneFormer: One Transformer to Rule Universal Image Segmentation](https://mlsanity.com/paper/2211.06220/) (2022)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [Convolutional Mesh Regression for Single-Image Human Shape Reconstruction](https://mlsanity.com/paper/1905.03244/) (2019)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [Neighborhood Attention Transformer](https://mlsanity.com/paper/2204.07143/) (2022)
- [Learning Joint Spatial-Temporal Transformations for Video Inpainting](https://mlsanity.com/paper/2007.10247/) (2020)
- [DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis](https://mlsanity.com/paper/2008.05865/) (2020)
- [MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2111.12707/) (2021)
- [Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction](https://mlsanity.com/paper/2103.12091/) (2021)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [Topological Planning with Transformers for Vision-and-Language Navigation](https://mlsanity.com/paper/2012.05292/) (2020)
- [Learning What and Where to Draw](https://mlsanity.com/paper/1610.02454/) (2016)
- [Multi-Channel Attention Selection GAN with Cascaded Semantic Guidance for Cross-View Image Translation](https://mlsanity.com/paper/1904.06807/) (2019)
- [House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation](https://mlsanity.com/paper/2003.06988/) (2020)

## Built on by (most cited first, 19 of 19)

- [AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes](https://mlsanity.com/paper/2312.06644/) (2023)
- [SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects](https://mlsanity.com/paper/2410.16499/) (2024)
- [Hierarchical Graph Interaction Transformer with Dynamic Token Clustering for Camouflaged Object Detection](https://mlsanity.com/paper/2408.15020/) (2024)
- [Gradformer: Graph Transformer with Exponential Decay](https://mlsanity.com/paper/2404.15729/) (2024)
- [ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents](https://mlsanity.com/paper/2510.21432/) (2025)
- [Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis](https://mlsanity.com/paper/2307.12084/) (2023)
- [Graph Transformer GANs with Graph Masked Modeling for Architectural Layout Generation](https://mlsanity.com/paper/2401.07721/) (2024)
- [ChatHouseDiffusion: Prompt-Guided Generation and Editing of Floor Plans](https://mlsanity.com/paper/2410.11908/) (2024)
- [Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis](https://mlsanity.com/paper/2507.06689/) (2025)
- [GenCO: Generating Diverse Designs with Combinatorial Constraints](https://mlsanity.com/paper/2310.02442/) (2023)
- [Computer-Aided Layout Generation for Building Design: A Review](https://mlsanity.com/paper/2504.09694/) (2025)
- [SSIG: A Visually-Guided Graph Edit Distance for Floor Plan Similarity](https://mlsanity.com/paper/2309.04357/) (2023)
- [Hierarchical Cross-Attention Network for Virtual Try-On](https://mlsanity.com/paper/2411.15542/) (2024)
- [Tokenization Allows Multimodal Large Language Models to Understand, Generate and Edit Architectural Floor Plans](https://mlsanity.com/paper/2603.11640/) (2026)
- [GFLAN: Generative Functional Layouts](https://mlsanity.com/paper/2512.16275/) (2025)
- [AdvSynGNN: Structure-Adaptive Graph Neural Nets via Adversarial Synthesis and Self-Corrective Propagation](https://mlsanity.com/paper/2602.17071/) (2026)
- [Rethinking Floorplan Generation with Space Syntax: From Design Theory to Model Improvement](https://mlsanity.com/paper/2602.22507/) (2026)
- [VoxScene: Anchor-Conditioned Voxel Diffusion for Indoor Scene Arrangement](https://mlsanity.com/paper/2605.17102/) (2026)
- [HypergraphFormer: Learning Hypergraphs from LLMs for Editable Floor Plan Generation](https://mlsanity.com/paper/2605.18932/) (2026)

## Abstract (arXiv)

We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task. The proposed graph-Transformer-based generator includes a novel graph Transformer encoder that combines graph convolutions and self-attentions in a Transformer to model both local and global interactions across connected and non-connected graph nodes. Specifically, the proposed connected node attention (CNA) and non-connected node attention (NNA) aim to capture the global relations across connected nodes and non-connected nodes in the input graph, respectively. The proposed graph modeling block (GMB) aims to exploit local vertex interactions based on a house layout topology. Moreover, we propose a new node classification-based discriminator to preserve the high-level semantic and discriminative node features for different house components. Finally, we propose a novel graph-based cycle-consistency loss that aims at maintaining the relative spatial relationships between ground truth and predicted graphs. Experiments on two challenging graph-constrained house generation tasks (i.e., house layout and roof generation) with two public datasets demonstrate the effectiveness of GTGAN in terms of objective quantitative scores and subjective visual realism. New state-of-the-art results are established by large margins on both tasks.