# From Model Uncertainty to Human Attention: Localization-Aware Visual Cues for Scalable Annotation Review

arXiv: https://arxiv.org/abs/2605.12303 · HTML: https://mlsanity.com/paper/2605.12303/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 41 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 27 of 27)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [A Comparative Survey of Deep Active Learning](https://mlsanity.com/paper/2203.13450/) (2022)
- [Extreme clicking for efficient object annotation](https://mlsanity.com/paper/1708.02750/) (2017)
- [Annotation-efficient deep learning for automatic medical image segmentation](https://mlsanity.com/paper/2012.04885/) (2020)
- [Active Learning for Deep Object Detection via Probabilistic Modeling](https://mlsanity.com/paper/2103.16130/) (2021)
- [Localization-Aware Active Learning for Object Detection](https://mlsanity.com/paper/1801.05124/) (2018)
- [Complementarity in Human-AI Collaboration: Concept, Sources, and Evidence](https://mlsanity.com/paper/2404.00029/) (2024)
- [Plug and Play Active Learning for Object Detection](https://mlsanity.com/paper/2211.11612/) (2022)
- [A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity](https://mlsanity.com/paper/2204.10806/) (2022)
- [Iterative Bounding Box Annotation for Object Detection](https://mlsanity.com/paper/2007.00961/) (2020)
- [Overcoming the Limitations of Localization Uncertainty: Efficient & Exact Non-Linear Post-Processing and Calibration](https://mlsanity.com/paper/2306.08981/) (2023)
- [Supporting Annotators with Affordances for Efficiently Labeling Conversational Data](https://mlsanity.com/paper/2403.07762/) (2024)
- [Cost-Sensitive Uncertainty-Based Failure Recognition for Object Detection](https://mlsanity.com/paper/2404.17427/) (2024)
- [Streamlining the Development of Active Learning Methods in Real-World Object Detection](https://mlsanity.com/paper/2508.19906/) (2025)
- [It's Not Just Labeling -- A Research on LLM Generated Feedback Interpretability and Image Labeling Sketch Features](https://mlsanity.com/paper/2505.19419/) (2025)
- [Data Annotation Quality Problems in AI-Enabled Perception System Development](https://mlsanity.com/paper/2511.16410/) (2025)

## Abstract (arXiv)

High-quality labeled data is essential for training robust machine learning models, yet obtaining annotations at scale remains expensive. AI-assisted annotation has therefore become standard in large-scale labeling workflows. However, in tasks where model predictions carry two independent components, a class label and spatial boundaries, a model may classify an object with high confidence while mislocalizing it. Existing AI-assisted workflows offer annotators no signal about where spatial errors are most likely. Without such guidance, humans may systematically underinspect subtly misplaced boxes. We address this by studying the effect of visualizing spatial uncertainty via a purpose-built interface. In a controlled study with 120 participants, those receiving uncertainty cues achieve higher label quality while being faster overall. A box-level analysis confirms that the cues redirect annotator effort toward high-uncertainty predictions and away from well-localized boxes. These findings establish localization uncertainty as a lever to improve human-in-the-loop annotation. Code is available at https://mos-ks.github.io/MUHA/.