# Investigating Positive and Negative Qualities of Human-in-the-Loop Optimization for Designing Interaction Techniques

arXiv: https://arxiv.org/abs/2204.07641 · HTML: https://mlsanity.com/paper/2204.07641/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 61 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 7 of 7)

- [Predictive Entropy Search for Multi-objective Bayesian Optimization](https://mlsanity.com/paper/1511.05467/) (2015)
- [Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators](https://mlsanity.com/paper/1901.06417/) (2019)
- [Sequential Gallery for Interactive Visual Design Optimization](https://mlsanity.com/paper/2005.04107/) (2020)
- [GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial Networks](https://mlsanity.com/paper/2101.09978/) (2021)
- [Optimizing User Interface Layouts via Gradient Descent](https://mlsanity.com/paper/2002.10702/) (2020)
- [AdaptiFont: Increasing Individuals' Reading Speed with a Generative Font Model and Bayesian Optimization](https://mlsanity.com/paper/2104.10741/) (2021)
- [Predicting Human Performance in Vertical Menu Selection Using Deep Learning](https://mlsanity.com/paper/1803.05073/) (2018)

## Built on by (most cited first, 6 of 6)

- [Cooperative Design Optimization through Natural Language Interaction](https://mlsanity.com/paper/2508.16077/) (2025)
- [Efficient Visual Appearance Optimization by Learning from Prior Preferences](https://mlsanity.com/paper/2507.15355/) (2025)
- [The Impact of Expertise in the Loop for Exploring Machine Rationality](https://mlsanity.com/paper/2302.05665/) (2023)
- [From Metrics to Meaning: Time to Rethink Evaluation in Human-AI Collaborative Design](https://mlsanity.com/paper/2402.07911/) (2024)
- [Cost-Aware Bayesian Optimization for Prototyping Interactive Devices](https://mlsanity.com/paper/2602.01774/) (2026)
- [Rethinking Opinion Measurement Interfaces for Human-in-the-Loop Optimization](https://mlsanity.com/paper/2610.14757/) (2026)

## Abstract (arXiv)

Designers reportedly struggle with design optimization tasks where they are asked to find a combination of design parameters that maximizes a given set of objectives. In HCI, design optimization problems are often exceedingly complex, involving multiple objectives and expensive empirical evaluations. Model-based computational design algorithms assist designers by generating design examples during design, however they assume a model of the interaction domain. Black box methods for assistance, on the other hand, can work with any design problem. However, virtually all empirical studies of this human-in-the-loop approach have been carried out by either researchers or end-users. The question stands out if such methods can help designers in realistic tasks. In this paper, we study Bayesian optimization as an algorithmic method to guide the design optimization process. It operates by proposing to a designer which design candidate to try next, given previous observations. We report observations from a comparative study with 40 novice designers who were tasked to optimize a complex 3D touch interaction technique. The optimizer helped designers explore larger proportions of the design space and arrive at a better solution, however they reported lower agency and expressiveness. Designers guided by an optimizer reported lower mental effort but also felt less creative and less in charge of the progress. We conclude that human-in-the-loop optimization can support novice designers in cases where agency is not critical.