# Optimizing User Interface Layouts via Gradient Descent

arXiv: https://arxiv.org/abs/2002.10702 · HTML: https://mlsanity.com/paper/2002.10702/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 6 of 6)

- [UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset](https://mlsanity.com/paper/2407.08850/) (2024)
- [Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques](https://mlsanity.com/paper/2204.07641/) (2022)
- [AdaptiFont: Increasing Individuals' Reading Speed with a Generative Font Model and Bayesian Optimization](https://mlsanity.com/paper/2104.10741/) (2021)
- [GreaseVision: Rewriting the Rules of the Interface](https://mlsanity.com/paper/2204.03731/) (2022)
- [The role of large language models in UI/UX design: A systematic literature review](https://mlsanity.com/paper/2507.04469/) (2025)
- [Designing Fatigue-Aware VR Interfaces via Biomechanical Models](https://mlsanity.com/paper/2603.26031/) (2026)

## Abstract (arXiv)

Automating parts of the user interface (UI) design process has been a longstanding challenge. We present an automated technique for optimizing the layouts of mobile UIs. Our method uses gradient descent on a neural network model of task performance with respect to the model's inputs to make layout modifications that result in improved predicted error rates and task completion times. We start by extending prior work on neural network based performance prediction to 2-dimensional mobile UIs with an expanded interaction space. We then apply our method to two UIs, including one that the model had not been trained on, to discover layout alternatives with significantly improved predicted performance. Finally, we confirm these predictions experimentally, showing improvements up to 9.2 percent in the optimized layouts. This demonstrates the algorithm's efficacy in improving the task performance of a layout, and its ability to generalize and improve layouts of new interfaces.