GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial Networks
Not on the map. Why?
Abstract (arXiv)
Graphical User Interface (GUI) is ubiquitous in almost all modern desktop software, mobile applications, and online websites. A good GUI design is crucial to the success of the software in the market, but designing a good GUI which requires much innovation and creativity is difficult even to well-trained designers. Besides, the requirement of the rapid development of GUI design also aggravates designers' working load. So, the availability of various automated generated GUIs can help enhance the design personalization and specialization as they can cater to the taste of different designers. To assist designers, we develop a model GUIGAN to automatically generate GUI designs. Different from conventional image generation models based on image pixels, our GUIGAN is to reuse GUI components collected from existing mobile app GUIs for composing a new design that is similar to natural-language generation. Our GUIGAN is based on SeqGAN by modeling the GUI component style compatibility and GUI structure. The evaluation demonstrates that our model significantly outperforms the best of the baseline methods by 30.77% in Frechet Inception distance (FID) and 12.35% in 1-Nearest Neighbor Accuracy (1-NNA). Through a pilot user study, we provide initial evidence of the usefulness of our approach for generating acceptable brand new GUI designs.
Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 26 of its 69 references that are in the map. First posted on arXiv 25 Jan 2021.
Builds on
26 of 69 references.
References · 26
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2016Improved Techniques for Training GANscited by 2,938
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
- 2017Wasserstein GANcited by 635
- 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
- 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
- 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
- 2016Revisiting Classifier Two-Sample Testscited by 258
- 2019LayoutVAE: Stochastic Scene Layout Generation From a Label Setcited by 119
14 more
- 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 113
- 2019Semantic Object Accuracy for Generative Text-to-Image Synthesiscited by 89
- 2018An empirical study on evaluation metrics of generative adversarial networkscited by 81
- 2020Attribute-conditioned Layout GAN for Automatic Graphic Designcited by 67
- 2018Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Appscited by 36
- 2020Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?cited by 29
- 2020Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learningcited by 25
- 2020Owl Eyes: Spotting UI Display Issues via Visual Understandingcited by 16
- 2019Controllable and Progressive Image Extrapolationcited by 13
- 2019StoryDroid: Automated Generation of Storyboard for Android Appscited by 8
- 2019Enabling Hyper-Personalisation: Automated Ad Creative Generation and Ranking for Fashion e-Commercecited by 8
- 2015Colors -Messengers of Concepts: Visual Design Mining for Learning Color Semanticscited by 7
- 2018Use of Neural Signals to Evaluate the Quality of Generative Adversarial Network Performance in Facial Image Generationcited by 7
- 2019Guigle: A GUI Search Engine for Android Appscited by 4
Built on by
Citing papers per year: 2021: 1; 2022: 6; 2023: 2; 2024: 3; 2025: 2; 2026: 3.
17 papers.
Sort:
By area:
- 2021Understanding Mobile GUI: from Pixel-Words to Screen-Sentencescited by 13
- 2025MP-GUI: Modality Perception with MLLMs for GUI Understandingcited by 12
- 2023Boosting GUI Prototyping with Diffusion Modelscited by 10
- 2022Generative Colorization of Structured Mobile Web Pagescited by 8
- 2024Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Modelcited by 7
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 7
- 2022Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniquescited by 5
- 2026GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environmentscited by 5
- 2022GreaseVision: Rewriting the Rules of the Interfacecited by 2
- 2022ULDGNN: A Fragmented UI Layer Detector Based on Graph Neural Networkscited by 2
- 2023GUILGET: GUI Layout GEneration with Transformercited by 2
- 2024GUI Element Detection Using SOTA YOLO Deep Learning Modelscited by 2
5 more
- 2025CANVAS: A Benchmark for Vision-Language Models on Tool-Based User Interface Designcited by 2
- 2026CoGen: Creation of Reusable UI Components in Figma via Textual Commandscited by 2
- 2022LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusingcited by 1
- 2022UI Layers Merger: Merging UI layers via Visual Learning and Boundary Priorcited by 1
- 2024Semantic GUI Scene Learning and Video Alignment for Detecting Duplicate Video-based Bug Reportscited by 1
- 2026GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments9 Feb 2026
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation5 Feb 2026
- 2026CoGen: Creation of Reusable UI Components in Figma via Textual Commands15 Jan 2026
- 2025CANVAS: A Benchmark for Vision-Language Models on Tool-Based User Interface Design25 Nov 2025
- 2025MP-GUI: Modality Perception with MLLMs for GUI Understanding18 Mar 2025
- 2024Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Model25 Sep 2024
- 2024GUI Element Detection Using SOTA YOLO Deep Learning Models7 Aug 2024
- 2024Semantic GUI Scene Learning and Video Alignment for Detecting Duplicate Video-based Bug Reports11 Jul 2024
- 2023Boosting GUI Prototyping with Diffusion Models9 Jun 2023
- 2023GUILGET: GUI Layout GEneration with Transformer18 Apr 2023
- 2022Generative Colorization of Structured Mobile Web Pages22 Dec 2022
- 2022ULDGNN: A Fragmented UI Layer Detector Based on Graph Neural Networks13 Aug 2022
5 more
- 2022UI Layers Merger: Merging UI layers via Visual Learning and Boundary Prior18 Jun 2022
- 2022Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques15 Apr 2022
- 2022LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusing8 Apr 2022
- 2022GreaseVision: Rewriting the Rules of the Interface7 Apr 2022
- 2021Understanding Mobile GUI: from Pixel-Words to Screen-Sentences25 May 2021
- 2021Understanding Mobile GUI: from Pixel-Words to Screen-Sentencescited by 13
- 2025MP-GUI: Modality Perception with MLLMs for GUI Understandingcited by 12
- 2022Generative Colorization of Structured Mobile Web Pagescited by 8
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 7
- 2022Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniquescited by 5
- 2026GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environmentscited by 5
- 2022GreaseVision: Rewriting the Rules of the Interfacecited by 2
- 2022ULDGNN: A Fragmented UI Layer Detector Based on Graph Neural Networkscited by 2
- 2023GUILGET: GUI Layout GEneration with Transformercited by 2
- 2024GUI Element Detection Using SOTA YOLO Deep Learning Modelscited by 2
- 2026CoGen: Creation of Reusable UI Components in Figma via Textual Commandscited by 2
- 2022LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusingcited by 1
12 most cited of 14 All 14 (JSON)
- 2026GUI-GenBench: Evaluating Image Generation Models as Interactive GUI Environments9 Feb 2026
- 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generation5 Feb 2026
- 2026CoGen: Creation of Reusable UI Components in Figma via Textual Commands15 Jan 2026
- 2025MP-GUI: Modality Perception with MLLMs for GUI Understanding18 Mar 2025
- 2024GUI Element Detection Using SOTA YOLO Deep Learning Models7 Aug 2024
- 2024Semantic GUI Scene Learning and Video Alignment for Detecting Duplicate Video-based Bug Reports11 Jul 2024
- 2023GUILGET: GUI Layout GEneration with Transformer18 Apr 2023
- 2022Generative Colorization of Structured Mobile Web Pages22 Dec 2022
- 2022ULDGNN: A Fragmented UI Layer Detector Based on Graph Neural Networks13 Aug 2022
- 2022UI Layers Merger: Merging UI layers via Visual Learning and Boundary Prior18 Jun 2022
- 2022Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques15 Apr 2022
- 2022LaF: Labeling-Free Model Selection for Automated Deep Neural Network Reusing8 Apr 2022
12 newest of 14 All 14 (JSON)