mlsanityData to 7 Oct 2026

Boosting GUI Prototyping with Diffusion Models

arXiv 2306.06233 ↗First posted 9 Jun 2023cs.SE cs.AI cs.CVcited by 10

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

GUI (graphical user interface) prototyping is a widely-used technique in requirements engineering for gathering and refining requirements, reducing development risks and increasing stakeholder engagement. However, GUI prototyping can be a time-consuming and costly process. In recent years, deep learning models such as Stable Diffusion have emerged as a powerful text-to-image tool capable of generating detailed images based on text prompts. In this paper, we propose UI-Diffuser, an approach that leverages Stable Diffusion to generate mobile UIs through simple textual descriptions and UI components. Preliminary results show that UI-Diffuser provides an efficient and cost-effective way to generate mobile GUI designs while reducing the need for extensive prototyping efforts. This approach has the potential to significantly improve the speed and efficiency of GUI prototyping in requirements engineering.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 25 of its 44 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

25 of 44 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023LayoutDM: Discrete Diffusion Model for Controllable Layout Generation14 Mar
  2. 2023Unifying Layout Generation with a Decoupled Diffusion Model9 Mar
  3. 2023DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformer7 Mar
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2023PLay: Parametrically Conditioned Layout Generation using Latent Diffusion27 Jan
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Generative Adversarial Networkscited by 5,603
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  8. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  10. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  11. 2021Screen2Vec: Semantic Embedding of GUI Screens and GUI Componentscited by 41
  12. 2021VINS: Visual Search for Mobile User Interface Designcited by 37
5 more
  1. 2021GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial Networkscited by 17
  2. 2021Wireframe-Based UI Design Search Through Image Autoencodercited by 17
  3. 2018Automated Reporting of GUI Design Violations for Mobile Appscited by 15
  4. 2022Gallery D.C.: Auto-created GUI Component Gallery for Design Search and Knowledge Discoverycited by 5
  5. 2019Guigle: A GUI Search Engine for Android Appscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 0; 2026: 4.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Zero-Shot Prompting Approaches for LLM-based Graphical User Interface Generation
  2. 2026From Natural Language Requirements to Graphical User Interfaces: Automated Prototyping and Verification with Pretrained Language Models ↗
  3. 2025GUIDE: LLM-Driven GUI Generation Decomposition for Automated Prototyping
  4. 2026AI Prototyper: A Figma Plugin for Decomposition-Based GUI Prototyping with LLMs ↗
  5. 2026Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding
  6. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents
  7. 2026UI Remix: Supporting UI Design Through Interactive Example Retrieval and Remixing ↗
  8. 2024CrowdGenUI: Aligning LLM-Based UI Generation with Crowdsourced User Preferences