# Black-box Uncertainty Quantification Method for LLM-as-a-Judge

arXiv: https://arxiv.org/abs/2410.11594 · HTML: https://mlsanity.com/paper/2410.11594/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 30 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Builds on (most cited first, 23 of 23)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models](https://mlsanity.com/paper/2405.01535/) (2024)
- [Towards Expert-Level Medical Question Answering with Large Language Models](https://mlsanity.com/paper/2305.09617/) (2023)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Prompting Large Language Model for Machine Translation: A Case Study](https://mlsanity.com/paper/2301.07069/) (2023)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Thermometer: Towards Universal Calibration for Large Language Models](https://mlsanity.com/paper/2403.08819/) (2024)
- [Finding Blind Spots in Evaluator LLMs with Interpretable Checklists](https://mlsanity.com/paper/2406.13439/) (2024)
- [Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment](https://mlsanity.com/paper/2204.03025/) (2022)
- [Large Language Model Confidence Estimation via Black-Box Access](https://mlsanity.com/paper/2406.04370/) (2024)

## Built on by (most cited first, 18 of 18)

- [Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworks](https://mlsanity.com/paper/2510.27106/) (2025)
- [Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction](https://mlsanity.com/paper/2509.18658/) (2025)
- [Entropy-Gated Branching for Efficient Test-Time Reasoning](https://mlsanity.com/paper/2503.21961/) (2025)
- [Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge](https://mlsanity.com/paper/2502.12501/) (2025)
- [Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theory](https://mlsanity.com/paper/2602.00521/) (2026)
- [Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations](https://mlsanity.com/paper/2508.03550/) (2025)
- [A Survey of Calibration Process for Black-Box LLMs](https://mlsanity.com/paper/2412.12767/) (2024)
- [VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation](https://mlsanity.com/paper/2604.25235/) (2026)
- [Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation](https://mlsanity.com/paper/2510.02279/) (2025)
- [Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models](https://mlsanity.com/paper/2506.09684/) (2025)
- [Calibration Is Not Enough: Evaluating Confidence Estimation Under Language Variations](https://mlsanity.com/paper/2601.08064/) (2026)
- [A Case Study Investigating the Role of Generative AI in Quality Evaluations of Epics in Agile Software Development](https://mlsanity.com/paper/2505.07664/) (2025)
- [Beyond statistical significance: Quantifying uncertainty and statistical variability in multilingual and multitask NLP evaluation](https://mlsanity.com/paper/2509.22612/) (2025)
- [Using Large Language Models for Legal Decision-Making in Austrian Value-Added Tax Law: A Comparative Study](https://mlsanity.com/paper/2507.08468/) (2025)
- [Enabling Fine-Grained Operating Points for Black-Box LLMs](https://mlsanity.com/paper/2510.17727/) (2025)
- [A Formal Framework for Uncertainty Analysis of Text Generation with Large Language Models](https://mlsanity.com/paper/2603.26363/) (2026)
- [S^3martCirc: Self-supervised Smart Circuit Discovery](https://mlsanity.com/paper/2609.00755/) (2026)
- [Robust Conformal Consensus: Multi-Agent LLM-as-a-Judge Interval Evaluation with Conformal Prediction](https://mlsanity.com/paper/2609.06367/) (2026)

## Abstract (arXiv)

LLM-as-a-Judge is a widely used method for evaluating the performance of Large Language Models (LLMs) across various tasks. We address the challenge of quantifying the uncertainty of LLM-as-a-Judge evaluations. While uncertainty quantification has been well-studied in other domains, applying it effectively to LLMs poses unique challenges due to their complex decision-making capabilities and computational demands. In this paper, we introduce a novel method for quantifying uncertainty designed to enhance the trustworthiness of LLM-as-a-Judge evaluations. The method quantifies uncertainty by analyzing the relationships between generated assessments and possible ratings. By cross-evaluating these relationships and constructing a confusion matrix based on token probabilities, the method derives labels of high or low uncertainty. We evaluate our method across multiple benchmarks, demonstrating a strong correlation between the accuracy of LLM evaluations and the derived uncertainty scores. Our findings suggest that this method can significantly improve the reliability and consistency of LLM-as-a-Judge evaluations.