mlsanityData to 7 Oct 2026

Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI

arXiv 2106.01410 ↗First posted 2 Jun 2021cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

In this paper, we describe an open source Python toolkit named Uncertainty Quantification 360 (UQ360) for the uncertainty quantification of AI models. The goal of this toolkit is twofold: first, to provide a broad range of capabilities to streamline as well as foster the common practices of quantifying, evaluating, improving, and communicating uncertainty in the AI application development lifecycle; second, to encourage further exploration of UQ's connections to other pillars of trustworthy AI such as fairness and transparency through the dissemination of latest research and education materials. Beyond the Python package (urlhttps://github.com/IBM/UQ360), we have developed an interactive experience (urlhttp://uq360.mybluemix.net) and guidance materials as educational tools to aid researchers and developers in producing and communicating high-quality uncertainties in an effective manner.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 9 of its 21 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

9 of 21 references.

References · 9

  1. 2020Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Makingcited by 281
  2. 2020Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertaintycited by 128
  3. 2017Model Selection in Bayesian Neural Networks via Horseshoe Priorscited by 52
  4. 2018Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probescited by 21
  5. 2019Building Calibrated Deep Models via Uncertainty Matching with Auxiliary Interval Predictorscited by 10
  6. 2020Approximate Cross-Validation for Structured Modelscited by 9
  7. 2020Learning Prediction Intervals for Model Performancecited by 9
  8. 2020Uncertainty Prediction for Deep Sequential Regression Using Meta Modelscited by 6
  9. 2021Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervalscited by 3

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 11; 2023: 2; 2024: 3; 2025: 3.

22 papers.

Sort:

  1. 2021Human-Centered Explainable AI (XAI): From Algorithms to User Experiencescited by 123
  2. 2021Towards a Science of Human-AI Decision Making: A Survey of Empirical Studiescited by 85
  3. 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
  4. 2022Connecting Algorithmic Research and Usage Contexts: A Perspective of Contextualized Evaluation for Explainable AIcited by 40
  5. 2024Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Makingcited by 38
  6. 2022Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkitscited by 33
  7. 2025As Confidence Aligns: Exploring the Effect of AI Confidence on Human Self-confidence in Human-AI Decision Makingcited by 18
  8. 2022JAWS: Auditing Predictive Uncertainty Under Covariate Shiftcited by 14
  9. 2023Fortuna: A Library for Uncertainty Quantification in Deep Learningcited by 14
  10. 2021TyXe: Pyro-based Bayesian neural nets for Pytorchcited by 6
  11. 2024Understanding the Effects of Miscalibrated AI Confidence on User Trust, Reliance, and Decision Efficacycited by 6
  12. 2022What is Flagged in Uncertainty Quantification? Latent Density Models for Uncertainty Categorizationcited by 5
10 more
  1. 2022A Survey on Uncertainty Toolkits for Deep Learningcited by 4
  2. 2022Quantitative AI Risk Assessments: Opportunities and Challengescited by 4
  3. 2022Tailored Uncertainty Estimation for Deep Learning Systemscited by 3
  4. 2022Explainable Global Error Weighted on Feature Importance: The xGEWFI metric to evaluate the error of data imputation and data augmentationcited by 1
  5. 2022Deep Surrogate of Modular Multi Pump using Active Learningcited by 1
  6. 2022Measuring the Confidence of Traffic Forecasting Models: Techniques, Experimental Comparison and Guidelines towards Their Actionabilitycited by 1
  7. 2023Function Composition in Trustworthy Machine Learning: Implementation Choices, Insights, and Questionscited by 1
  8. 2024A Decision-driven Methodology for Designing Uncertainty-aware AI Self-Assessmentcited by 1
  9. 2025Neurosymbolic Artificial Intelligence for Robust Network Intrusion Detection: From Scratch to Transfer Learningcited by 1
  10. 2025Bridging Ethical Principles and Algorithmic Methods: An Alternative Approach for Assessing Trustworthiness in AI Systemscited by 1