# A Study on Mitigating Hard Boundaries of Decision-Tree-based Uncertainty Estimates for AI Models

arXiv: https://arxiv.org/abs/2201.03263 · HTML: https://mlsanity.com/paper/2201.03263/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 29 references that are in the map.
- First posted on arXiv 10 Jan 2022.

## Builds on (most cited first, 6 of 6)

- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Comparison of Uncertainty Estimation Approaches in Deep Learning Components for Autonomous Vehicle Applications](https://mlsanity.com/paper/2006.15172/) (2020)
- [Towards Identifying and Managing Sources of Uncertainty in AI and Machine Learning Models - An Overview](https://mlsanity.com/paper/1811.11669/) (2018)

## Built on by (most cited first, 2 of 2)

- [Architectural patterns for handling runtime uncertainty of data-driven models in safety-critical perception](https://mlsanity.com/paper/2206.06838/) (2022)
- [Uncertainty Wrapper in the medical domain: Establishing transparent uncertainty quantification for opaque machine learning models in practice](https://mlsanity.com/paper/2311.05245/) (2023)

## Abstract (arXiv)

Outcomes of data-driven AI models cannot be assumed to be always correct. To estimate the uncertainty in these outcomes, the uncertainty wrapper framework has been proposed, which considers uncertainties related to model fit, input quality, and scope compliance. Uncertainty wrappers use a decision tree approach to cluster input quality related uncertainties, assigning inputs strictly to distinct uncertainty clusters. Hence, a slight variation in only one feature may lead to a cluster assignment with a significantly different uncertainty. Our objective is to replace this with an approach that mitigates hard decision boundaries of these assignments while preserving interpretability, runtime complexity, and prediction performance. Five approaches were selected as candidates and integrated into the uncertainty wrapper framework. For the evaluation based on the Brier score, datasets for a pedestrian detection use case were generated using the CARLA simulator and YOLOv3. All integrated approaches achieved a softening, i.e., smoothing, of uncertainty estimation. Yet, compared to decision trees, they are not so easy to interpret and have higher runtime complexity. Moreover, some components of the Brier score impaired while others improved. Most promising regarding the Brier score were random forests. In conclusion, softening hard decision tree boundaries appears to be a trade-off decision.