Portal UX Agent -- A Plug-and-Play Engine for Rendering UIs from Natural Language Specifications
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
The rapid appearance of large language models (LLMs) has led to systems that turn natural-language intent into real user interfaces (UIs). Free-form code generation maximizes expressiveness but often hurts reliability, security, and design-system compliance. In contrast, fully static UIs are easy to govern but lack adaptability. We present the Portal UX Agent, a practical middle way that makes bounded generation work: an LLM plans the UI at a high level, and a deterministic renderer assembles the final interface from a vetted set of components and layout templates. The agent maps intents to a typed composition-template and component specifications-constrained by a schema. This enables auditability, reuse, and safety while preserving flexibility. We also introduce a mixed-methods evaluation framework that combines automatic checks (coverage, property fidelity, layout, accessibility, performance) with an LLM-as-a-Judge rubric to assess semantic alignment and visual polish. Experiments on multi-domain portal scenarios show that the Portal UX Agent reliably turns intent into coherent, usable UIs and performs well on compositionality and clarity. This work advances agentic UI design by combining model-driven representations, plug-and-play rendering, and structured evaluation, paving the way for controllable and trustworthy UI generation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 5 of its 8 references that are in the map. First posted on arXiv 2 Nov 2025.
Builds on
5 of 8 references.
References · 5
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2020Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthycited by 182
- 2024Generative AI for Visualization: State of the Art and Future Directionscited by 23
- 2021Image2Lego: Customized LEGO Set Generation from Imagescited by 14
- 2025SpecifyUI: Supporting Iterative UI Design Intent Expression through Structured Specifications and Generative AIcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025WebVIA: A Web-based Vision-Language Agentic Framework for Interactive and Verifiable UI-to-Code Generation
- 2025Computer-Use Agents as Judges for Generative User Interface
- 2026Enabling Creative Exploration for Vibe Design Agents
- 2025From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents
- 2025ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents
- 2025VSA:Visual-Structural Alignment for UI-to-Code
- 2024UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback
- 2026VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents