# Automating UI Optimization through Multi-Agentic Reasoning

arXiv: https://arxiv.org/abs/2602.13126 · HTML: https://mlsanity.com/paper/2602.13126/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 71 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction](https://mlsanity.com/paper/2305.06474/) (2023)
- [Enabling Conversational Interaction with Mobile UI using Large Language Models](https://mlsanity.com/paper/2209.08655/) (2022)
- [Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructions](https://mlsanity.com/paper/1710.06280/) (2017)
- [Aligning Language Models to Explicitly Handle Ambiguity](https://mlsanity.com/paper/2404.11972/) (2024)
- [VisionTasker: Mobile Task Automation Using Vision Based UI Understanding and LLM Task Planning](https://mlsanity.com/paper/2312.11190/) (2023)
- [SituationAdapt: Contextual UI Optimization in Mixed Reality with Situation Awareness via LLM Reasoning](https://mlsanity.com/paper/2409.12836/) (2024)
- [Preference-Guided Multi-Objective UI Adaptation](https://mlsanity.com/paper/2509.18960/) (2025)
- [Computational Workflows for Designing Input Devices](https://mlsanity.com/paper/2101.10032/) (2021)
- [CrowdGenUI: Aligning LLM-Based UI Generation with Crowdsourced User Preferences](https://mlsanity.com/paper/2411.03477/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Benchmarking Framework for Context-aware XR Interfaces](https://mlsanity.com/paper/2609.30466/) (2026)

## Abstract (arXiv)

We present AutoOptimization, a novel multi-objective optimization framework for adapting user interfaces. From a user's verbal preferences for changing a UI, our framework guides a prioritization-based Pareto frontier search over candidate layouts. It selects suitable objective functions for UI placement while simultaneously parameterizing them according to the user's instructions to define the optimization problem. A solver then generates a series of optimal UI layouts, which our framework validates against the user's instructions to adapt the UI with the final solution. Our approach thus overcomes the previous need for manual inspection of layouts and the use of population averages for objective parameters. We integrate multiple agents sequentially within our framework, enabling the system to leverage their reasoning capabilities to interpret user preferences, configure the optimization problem, and validate optimization outcomes.