mlsanityData to 7 Oct 2026

From Predictions to Decisions: The Importance of Joint Predictive Distributions

arXiv 2107.09224 ↗First posted 20 Jul 2021cs.LG stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

A fundamental challenge for any intelligent system is prediction: given some inputs, can you predict corresponding outcomes? Most work on supervised learning has focused on producing accurate marginal predictions for each input. However, we show that for a broad class of decision problems, accurate joint predictions are required to deliver good performance. In particular, we establish several results pertaining to combinatorial decision problems, sequential predictions, and multi-armed bandits to elucidate the essential role of joint predictive distributions. Our treatment of multi-armed bandits introduces an approximate Thompson sampling algorithm and analytic techniques that lead to a new kind of regret bound.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 12 of its 27 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

12 of 27 references.

References · 12

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  3. 2017A Tutorial on Thompson Samplingcited by 420
  4. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  5. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  6. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  7. 2017Ensemble Samplingcited by 62
  8. 2021Reinforcement Learning, Bit by Bitcited by 59
  9. 2015Bootstrapped Thompson Sampling and Deep Explorationcited by 40
  10. 2020Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?cited by 21
  11. 2019Thompson Sampling with Approximate Inferencecited by 15
  12. 2019Scalable Thompson Sampling via Optimal Transportcited by 7

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 8; 2023: 4; 2024: 8; 2025: 7.

28 papers.

Sort:

By area:

  1. 2024To Believe or Not to Believe Your LLMcited by 120
  2. 2024Position: Bayesian Deep Learning is Needed in the Age of Large-Scale AIcited by 60
  3. 2022Bayesian Model Selection, the Marginal Likelihood, and Generalizationcited by 56
  4. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 40
  5. 2023Approximate Thompson Sampling via Epistemic Neural Networkscited by 31
  6. 2022An Analysis of Ensemble Samplingcited by 23
  7. 2021The Neural Testbed: Evaluating Joint Predictionscited by 22
  8. 2022Fine-Tuning Language Models via Epistemic Neural Networkscited by 22
  9. 2022Non-Stationary Bandit Learning via Predictive Samplingcited by 21
  10. 2022Ensembles for Uncertainty Estimation: Benefits of Prior Functions and Bootstrappingcited by 19
  11. 2024Experts Don't Cheat: Learning What You Don't Know By Predicting Pairscited by 18
  12. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
16 more
  1. 2024On Subjective Uncertainty Quantification and Calibration in Natural Language Generationcited by 14
  2. 2022Evaluating High-Order Predictive Distributions in Deep Learningcited by 10
  3. 2023Promises and Pitfalls of the Linearized Laplace in Bayesian Optimizationcited by 10
  4. 2024Provable Uncertainty Decomposition via Higher-Order Calibrationcited by 8
  5. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  6. 2025Adaptive Elicitation of Latent Information Using Natural Languagecited by 7
  7. 2025Contextual Thompson Sampling via Generation of Missing Datacited by 5
  8. 2025Efficiently Generating Correlated Sample Paths from Multi-step Time Series Foundation Modelscited by 4
  9. 2022Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Samplingcited by 3
  10. 2023Information-theoretic Analysis of Test Data Sensitivity in Uncertaintycited by 3
  11. 2025Architectural and Inferential Inductive Biases For Exchangeable Sequence Modelingcited by 2
  12. 2025SPIEDiff: robust learning of long-time macroscopic dynamics from short-time particle simulations with quantified epistemic uncertaintycited by 2
  13. 2023PICProp: Physics-Informed Confidence Propagation for Uncertainty Quantificationcited by 1
  14. 2025Improved Exploration in GFlownets via Enhanced Epistemic Neural Networkscited by 1
  15. 2022Robustness of Epinets against Distributional Shiftscited by 0
  16. 2025Prediction-Oriented Subsampling from Data Streamscited by 0