# Every pooling rule has its world: matching probability combination rules to situations and stakes

arXiv: https://arxiv.org/abs/2608.11275 · HTML: https://mlsanity.com/paper/2608.11275/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 80 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Builds on (most cited first, 21 of 21)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [AI safety via debate](https://mlsanity.com/paper/1805.00899/) (2018)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Calibrating Large Language Models with Sample Consistency](https://mlsanity.com/paper/2402.13904/) (2024)
- [Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network](https://mlsanity.com/paper/1711.08324/) (2017)
- [Asymptotically Exact, Embarrassingly Parallel MCMC](https://mlsanity.com/paper/1311.4780/) (2013)
- [Of Quantiles and Expectiles: Consistent Scoring Functions, Choquet Representations, and Forecast Rankings](https://mlsanity.com/paper/1503.08195/) (2015)
- [Combining Confidence Elicitation and Sample-based Methods for Uncertainty Quantification in Misinformation Mitigation](https://mlsanity.com/paper/2401.08694/) (2024)
- [Decision-Making with Belief Functions: a Review](https://mlsanity.com/paper/1808.05322/) (2018)
- [Model Comparison and Calibration Assessment: User Guide for Consistent Scoring Functions in Machine Learning and Actuarial Practice](https://mlsanity.com/paper/2202.12780/) (2022)
- [Probabilistic Interpretations for MYCIN's Certainty Factors](https://mlsanity.com/paper/1304.3419/) (2013)
- [Structure-Preserving Uncertainty Propagation in First-Order Proof Search](https://mlsanity.com/paper/2608.09190/) (2026)

## Abstract (arXiv)

Systems often need to combine two numerical assessments of the same yes/no question. The appropriate formula depends on what the numbers represent and on how the sources are related. Averaging is correct when one of several alternative interpretations applies; multiplying odds is correct when probability reports are based on conditionally independent evidence and a common prior; and probabilities of alternative successful derivations require their dependence or shared evidence to be taken into account. We state the assumptions behind several common combination rules and derive the corresponding combined probabilities. Two groups of Monte Carlo experiments address different questions. First, controlled generating mechanisms verify that the derived rule recovers the correct probability in the situations for which its assumptions hold. Second, the same mechanisms measure the consequences of using a mismatched rule, using logarithmic score and threshold decisions with different costs. Distinct pooling rules can produce the same binary decision at threshold 1/2 while assigning substantially different probabilities, so binary accuracy alone can conceal important differences. We also give probabilistic interpretations of conflicting-evidence rules and show that, for overlapping derivations, retaining the identities of shared uncertain premises permits direct calculation of the probability that at least one derivation is available. Pairwise combination of proof probabilities loses information when there are three or more derivations.