mlsanityData to 14 Oct 2026

Design Theater: Evaluating the Gap Between User-Facing Design Reasoning and Implementation in Generative UI Tools

arXiv 2607.22928 ↗First posted 24 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Generative UI tools promise to democratize UI design by turning natural language descriptions into complete interfaces. Alongside the interface, these tools generate user-facing design rationales that explain their layout, accessibility, and design choices. However, it remains unclear whether these stated rationales are actually reflected in the interfaces they produce. We call this disconnect ``Design Theater'': plausible and confident design rationales that have little relationship to the actual implementation. To study this phenomenon, we introduce a benchmark and three metrics for measuring Design Theater. The benchmark includes 24 UI generation tasks spanning structural, styling, and functional design requirements. Using this benchmark, we evaluate 120 interfaces created by five generative UI tools. On average, over 25% of user-facing design rationales are not implemented in the generated interface, and the implementation failure increases to 34% for functional requirements. Tools recognize roughly half of the UX principles embedded in prompts (mean = 0.54), with four of five tools implementing 6% or fewer functional principles. We also measure interface similarity across tools and find convergence in visual appearance and layout organization, with greater variation in color choices. Overall, we contribute: 1) the concept of Design Theater; 2) a benchmark with metrics for assessing whether the stated reasoning of generative UI tools is reflected in their implementations; 3) and findings from a systematic evaluation of these tools. We discuss what these findings mean for the design and evaluation of generative UI tools.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 65 references that are in the map. First posted on arXiv 24 Jul 2026.

Builds on

25 of 65 references.

References · 25

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  4. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  5. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  6. 2023Generative artificial intelligence enhances creativity but reduces the diversity of novel contentcited by 167
  7. 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
  8. 2017pix2code: Generating Code from a Graphical User Interface Screenshotcited by 114
  9. 2022Designing for Responsible Trust in AI Systems: A Communication Perspectivecited by 48
  10. 2023Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Studycited by 38
  11. 2024The Effects of Generative AI on Design Fixation and Divergent Thinkingcited by 37
  12. 2021Screen Parsing: Towards Reverse Engineering of UI Models from Screenshotscited by 32
13 more
  1. 2025Vibe Coding in Practice: Motivations, Challenges, and a Future Outlook -- a Grey Literature Reviewcited by 30
  2. 2024Towards interactive evaluations for interaction harms in human-AI systemscited by 29
  3. 2025Vibe coding: programming through conversation with artificial intelligencecited by 29
  4. 2019Sketch2Code: Transformation of Sketches to UI in Real-time Using Deep Neural Networkcited by 23
  5. 2020GUIComp: A GUI Design Assistant with Real-Time, Multi-Faceted Feedbackcited by 11
  6. 2023Never-ending Learning of User Interfacescited by 9
  7. 2025The GenUI Study: Exploring the Design of Generative UI Tools to Support UX Practitioners and Beyondcited by 9
  8. 2022Trustworthy Social Bias Measurementcited by 7
  9. 2025DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Designcited by 6
  10. 2025Vibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaborationcited by 4
  11. 2025ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inferencecited by 4
  12. 2026Co-Designing Collaborative Generative AI Tools for Freelancerscited by 3
  13. 2026Seeing the Reasoning: How LLM Rationales Influence User Trust and Decision-Making in Factual Verification Taskscited by 1

Built on by

Not cited yet.