# UQLM: A Python Package for Uncertainty Quantification in Large Language Models

arXiv: https://arxiv.org/abs/2507.06196 · HTML: https://mlsanity.com/paper/2507.06196/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [AlignScore: Evaluating Factual Consistency with a Unified Alignment Function](https://mlsanity.com/paper/2305.16739/) (2023)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://mlsanity.com/paper/2307.13528/) (2023)
- [LM-Polygraph: Uncertainty Estimation for Language Models](https://mlsanity.com/paper/2311.07383/) (2023)
- [Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness](https://mlsanity.com/paper/2308.16175/) (2023)
- [A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions](https://mlsanity.com/paper/2412.05563/) (2024)
- [Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph](https://mlsanity.com/paper/2406.15627/) (2024)
- [Selectively Answering Ambiguous Questions](https://mlsanity.com/paper/2305.14613/) (2023)
- [Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency](https://mlsanity.com/paper/2502.04964/) (2025)
- [RefChecker: Reference-based Fine-grained Hallucination Checker and Benchmark for Large Language Models](https://mlsanity.com/paper/2405.14486/) (2024)
- [Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers](https://mlsanity.com/paper/2504.19254/) (2025)
- [RED-CT: A Systems Design Methodology for Using LLM-labeled Data to Train and Deploy Edge Classifiers for Computational Social Science](https://mlsanity.com/paper/2408.08217/) (2024)

## Built on by (most cited first, 9 of 9)

- [Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoning](https://mlsanity.com/paper/2509.02401/) (2025)
- [Fine-Grained Uncertainty Quantification for Long-Form Language Model Outputs: A Comparative Study](https://mlsanity.com/paper/2602.17431/) (2026)
- [Is Escalation Worth It? On the Depth of LLM Cascades](https://mlsanity.com/paper/2605.06350/) (2026)
- [Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training](https://mlsanity.com/paper/2605.02241/) (2026)
- [Sanity Checks for Long-Form Hallucination Detection](https://mlsanity.com/paper/2605.08346/) (2026)
- [Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation](https://mlsanity.com/paper/2607.02182/) (2026)
- [Unsupervised Evaluation of Multi-Turn Objective-Driven Interactions](https://mlsanity.com/paper/2511.03047/) (2025)
- [Silicon Bureaucracy and AI Test-Oriented Education: Contamination Sensitivity and Score Confidence in LLM Benchmarks](https://mlsanity.com/paper/2603.21636/) (2026)
- [How Do Uncertainty Estimators Align with Hallucination Occurrence in LLMs?](https://mlsanity.com/paper/2605.27016/) (2026)

## Abstract (arXiv)

Hallucinations, defined as instances where Large Language Models (LLMs) generate false or misleading content, pose a significant challenge that impacts the safety and trust of downstream applications. We introduce UQLM, a Python package for LLM hallucination detection using state-of-the-art uncertainty quantification (UQ) techniques. This toolkit offers a suite of UQ-based scorers that compute response-level confidence scores ranging from 0 to 1. This library provides an off-the-shelf solution for UQ-based hallucination detection that can be easily integrated to enhance the reliability of LLM outputs.