# Vision: "What Don't I Know About User's Preferences?" How Uncertainty Estimation Improves Proactive Assistant Alignment

arXiv: https://arxiv.org/abs/2610.15948 · HTML: https://mlsanity.com/paper/2610.15948/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 11 Oct 2026.

## Builds on (most cited first, 16 of 16)

- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [Adaptive Conformal Prediction for Motion Planning among Dynamic Agents](https://mlsanity.com/paper/2212.00278/) (2022)
- [Towards Human-centered Proactive Conversational Agents](https://mlsanity.com/paper/2404.12670/) (2024)
- [Role of Human-AI Interaction in Selective Prediction](https://mlsanity.com/paper/2112.06751/) (2021)
- [Collaborative Uncertainty in Multi-Agent Trajectory Forecasting](https://mlsanity.com/paper/2110.13947/) (2021)
- [Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery](https://mlsanity.com/paper/2003.01753/) (2020)
- [A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control](https://mlsanity.com/paper/2407.12195/) (2024)
- [Probabilistic Human Motion Prediction via A Bayesian Neural Network](https://mlsanity.com/paper/2107.06564/) (2021)
- [Adaptive Activity Monitoring with Uncertainty Quantification in Switching Gaussian Process Models](https://mlsanity.com/paper/1901.02427/) (2019)
- [Disentangling Uncertainties by Learning Compressed Data Representation](https://mlsanity.com/paper/2503.15801/) (2025)
- [Design and Evaluation of Generative Agent-based Platform for Human-Assistant Interaction Research: A Tale of 10 User Studies](https://mlsanity.com/paper/2505.09938/) (2025)
- [Beyond Noisy-TVs: Noise-Robust Exploration Via Learning Progress Monitoring](https://mlsanity.com/paper/2509.25438/) (2025)
- [PACT: Proactive Asking for Continual Task Assistance in Human-Robot Collaboration](https://mlsanity.com/paper/2605.24350/) (2026)
- [Personalized and Robust Proactive Robot Assistance with Uncertainty-Guided LLM Reasoning](https://mlsanity.com/paper/2606.08458/) (2026)

## Abstract (arXiv)

Proactive Smart Assistants (PSAs) increasingly use general knowledge, learned patterns from group-level data, patterns from individual user's data, and recent sensor observations to anticipate which action a user may prefer. Existing systems typically select actions from predicted user preference likelihoods, but these likelihoods alone do not reveal how reliable the prediction is, which evidence supports it, or why ambiguity remains. In this vision paper, we argue that uncertainty estimation can provide this missing information and substantially change appropriate downstream behavior. Using the example of a home assistant, we introduce the questions that uncertainty estimation aims to answer, i.e. how sure is the assistant about the predicted preference likelihood, whether the confidence come from group-level assumptions or individual's data, and whether likelihood ambiguity reflect the user's intrinsic preference variability or insufficient data for the particular situation. We further introduce uncertainty estimation applications in neighboring domains, such as human trajectory prediction, personalized activity recognition, information-collection, and model-based control. Finally, we draw on insights from neighboring domains to envision how future uncertainty-aware proactive assistants may address key challenges in proactive interaction.