# Building Intelligent User Interfaces for Human-AI Alignment

arXiv: https://arxiv.org/abs/2602.11753 · HTML: https://mlsanity.com/paper/2602.11753/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 11 of 11)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Scalable agent alignment via reward modeling: a research direction](https://mlsanity.com/paper/1811.07871/) (2018)
- [Interactive AI Alignment: Specification, Process, and Evaluation Alignment](https://mlsanity.com/paper/2311.00710/) (2023)
- [RLHF-Blender: A Configurable Interactive Interface for Learning from Diverse Human Feedback](https://mlsanity.com/paper/2308.04332/) (2023)
- [Reward Learning from Multiple Feedback Types](https://mlsanity.com/paper/2502.21038/) (2025)
- [Time-Efficient Reward Learning via Visually Assisted Cluster Ranking](https://mlsanity.com/paper/2212.00169/) (2022)
- [Interactive Groupwise Comparison for Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2507.04340/) (2025)
- [DxHF: Providing High-Quality Human Feedback for LLM Alignment via Interactive Decomposition](https://mlsanity.com/paper/2507.18802/) (2025)
- [Human-AI Complementarity: A Goal for Amplified Oversight](https://mlsanity.com/paper/2510.26518/) (2025)

## Abstract (arXiv)

Aligning AI systems with human values fundamentally relies on effective human feedback. While significant research has addressed training algorithms, the role of user interface is often overlooked and only treated as an implementation detail rather than a critical factor of alignment. This paper addresses this gap by introducing a reference model that offers a systematic framework for analyzing where and how user interface contributions can improve human-AI alignment. The structured taxonomy of the reference model is demonstrated through two case studies and a preliminary investigation featuring six user interfaces. This work highlights opportunities to advance alignment through human-computer interaction.