mlsanityData to 14 Oct 2026

Cross-Domain Uncertainty Quantification for Selective Prediction: A Comprehensive Bound Ablation with Transfer-Informed Betting

arXiv 2603.08907 ↗First posted 9 Mar 2026cs.LG cs.AI stat.MLcited by 1

Not on the map. Why?

Abstract (arXiv)

We present a comprehensive ablation of nine finite-sample bound families for selective prediction with risk control, combining concentration inequalities (Hoeffding, Empirical Bernstein, Clopper-Pearson, Wasserstein DRO, CVaR) with multiple-testing corrections (union bound, Learn Then Test fixed-sequence) and betting-based confidence sequences (WSR). Our main theoretical contribution is Transfer-Informed Betting (TIB), which warm-starts the WSR wealth process using a source domain's risk profile, achieving tighter bounds in data-scarce settings with a formal dominance guarantee. We prove that the TIB wealth process remains a valid supermartingale under all source-target divergences, that TIB dominates standard WSR when domains match, and that no data-independent warm-start can achieve better convergence. The combination of betting-based confidence sequences, LTT monotone testing, and cross-domain transfer is, to our knowledge, a three-way novelty not present in the literature. We evaluate all nine bound families on four benchmarks-MASSIVE (n=1,102), NyayaBench (n=280), CLINC-150 (n=22.5K), and Banking77 (n=13K)-across 18 (alpha, delta) configurations. On MASSIVE at alpha=0.10, LTT eliminates the ln(K) union-bound penalty, achieving 94.0% guaranteed coverage versus 73.8% for Hoeffding-a 27% relative improvement. On NyayaBench, where the small calibration set makes Hoeffding-family bounds infeasible below alpha=0.20, Transfer-Informed Betting achieves 18.5% coverage at alpha=0.10, a 5.4x improvement over LTT + Hoeffding. We additionally compare with split-conformal prediction, showing that conformal methods produce prediction sets (avg. 1.67 classes) whereas selective prediction provides single-prediction risk guarantees. We apply these methods to agentic caching systems, formalizing a progressive trust model where the guarantee determines when cached responses can be served autonomously.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

16 of 23 references.

References · 16

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2017Selective Classification for Deep Neural Networkscited by 852
  3. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  4. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  5. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
  6. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 271
  7. 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
  8. 2020Estimating means of bounded random variables by bettingcited by 169
  9. 2022MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset with 51 Typologically-Diverse Languagescited by 146
  10. 2022Efficient Few-Shot Learning Without Promptscited by 137
  11. 2024RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generationcited by 98
  12. 2021There are natural scores: Full comment on Shafer, "Testing by betting: A strategy for statistical and scientific communication"cited by 53
4 more
  1. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 27
  2. 2025vCache: Verified Semantic Prompt Cachingcited by 17
  3. 2025Advancing Semantic Caching for LLMs with Domain-Specific Embeddings and Synthetic Datacited by 7
  4. 2026SemanticALLI: Caching Reasoning, Not Just Responses, in Agentic Systemscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026FinAbstain: Uncertainty-Calibrated Multimodal RAG for Selective Financial Forecasting
  2. 2026Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation
  3. 2025Second-Order Bounds for [0,1]-Valued Regression via Betting Loss
  4. 2023Risk Aware Benchmarking of Large Language Models
  5. 2026Available Guardrails: Certifying Selective Prediction across ML Systems
  6. 2026Taming the Greeks: Option Portfolios with Inductive Biases
  7. 2026Learning to Bet for Horizon-Aware Anytime-Valid Testing
  8. 2026FinBench: Time-Gated Calibration and Uncertainty Benchmarking for Agentic Financial Forecasting