# Adapting User Interfaces with Model-based Reinforcement Learning

arXiv: https://arxiv.org/abs/2103.06807 · HTML: https://mlsanity.com/paper/2103.06807/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 62 references that are in the map.
- First posted on arXiv 11 Mar 2021.

## Builds on (most cited first, 5 of 5)

- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Model-Based Reinforcement Learning for Atari](https://mlsanity.com/paper/1903.00374/) (2019)
- [Modeling Human Visual Search Performance on Realistic Webpages Using Analytical and Deep Learning Methods](https://mlsanity.com/paper/2005.03784/) (2020)
- [SAM: A Modular Framework for Self-Adapting Web Menus](https://mlsanity.com/paper/1901.08289/) (2019)
- [Optimal Sensor Position for a Computer Mouse](https://mlsanity.com/paper/2001.03352/) (2020)

## Built on by (most cited first, 17 of 17)

- [OpenAgents: An Open Platform for Language Agents in the Wild](https://mlsanity.com/paper/2310.10634/) (2023)
- [Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications](https://mlsanity.com/paper/2303.08060/) (2023)
- [Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning](https://mlsanity.com/paper/2403.05911/) (2024)
- [MUG: Interactive Multimodal Grounding on User Interfaces](https://mlsanity.com/paper/2209.15099/) (2022)
- [When Should Users Check? Modeling Confirmation Frequency in Multi-Step Agentic AI Tasks](https://mlsanity.com/paper/2510.05307/) (2025)
- [Rediscovering Affordance: A Reinforcement Learning Perspective](https://mlsanity.com/paper/2112.12886/) (2021)
- [Composable Prompting Workspaces for Creative Writing: Exploration and Iteration Using Dynamic Widgets](https://mlsanity.com/paper/2503.21394/) (2025)
- [Simulation-Based Optimization of User Interfaces for Quality-Assuring Machine Learning Model Predictions](https://mlsanity.com/paper/2104.01129/) (2021)
- [PromptCanvas: Composable Prompting Workspaces Using Dynamic Widgets for Exploration and Iteration in Creative Writing](https://mlsanity.com/paper/2506.03741/) (2025)
- [Adaptive 3D UI Placement in Mixed Reality Using Deep Reinforcement Learning](https://mlsanity.com/paper/2504.21731/) (2025)
- [Computational Adaptation of XR Interfaces Through Interaction Simulation](https://mlsanity.com/paper/2204.09162/) (2022)
- [Integrating Human Feedback into a Reinforcement Learning-Based Framework for Adaptive User Interfaces](https://mlsanity.com/paper/2504.20782/) (2025)
- [Enhanced Web User Interface Design Via Cross-Device Responsiveness Assessment Using An Improved HCI-INTEGRATED DL Schemes](https://mlsanity.com/paper/2512.15775/) (2025)
- [Designing Fatigue-Aware VR Interfaces via Biomechanical Models](https://mlsanity.com/paper/2603.26031/) (2026)
- [Augmenting Human Performance with an XR Agent Learning from Online Behavior and BCI Evidence](https://mlsanity.com/paper/2608.30369/) (2026)
- [A Benchmarking Framework for Context-aware XR Interfaces](https://mlsanity.com/paper/2609.30466/) (2026)
- [Learning to Clarify Underspecified Intents Under Limited Interaction](https://mlsanity.com/paper/2610.04719/) (2026)

## Abstract (arXiv)

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user -- for example, due to surprise or relearning effort -- or "trap" the process to a suboptimal design immaturely. However, effects on users are hard to predict as they depend on factors that are latent and evolve over the course of interaction. We propose a novel approach for adaptive user interfaces that yields a conservative adaptation policy: It finds beneficial changes when there are such and avoids changes when there are none. Our model-based reinforcement learning method plans sequences of adaptations and consults predictive HCI models to estimate their effects. We present empirical and simulation results from the case of adaptive menus, showing that the method outperforms both a non-adaptive and a frequency-based policy.