mlsanityData to 14 Oct 2026

Tail-Influence Sampling for CVaR Policy Evaluation

arXiv 2609.38096 ↗First posted 29 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Policies with similar mean returns can differ sharply in rare failures, yet estimating lower-tail conditional value-at-risk (CVaR) accurately can require many costly rollouts. When different conditional components of a stochastic workflow can be queried separately, we ask how to allocate a fixed evaluation budget to estimate a fixed policy's CVaR most accurately. We derive a tail influence for each queryable conditional law that aggregates how its uncertainty affects CVaR across every Bellman reuse. Its variance yields the fixed-design efficiency bound and the oracle Neyman allocation. Tail-Influence Sampling (TIS) estimates these influence scales from a pilot model and reallocates fresh queries toward kernels that matter most for the tail; a visitation-anchored variant protects against pilot underallocation. Under fixed dimension and a positive quantile margin, TIS attains oracle asymptotic variance and first-order MSE including pilot cost, while the anchored variant is within a factor two of the oracle. We also characterize an exact-grid regime in which tail- and mean-optimal allocations coincide. On CliffWalking, TIS reduces MSE by 41% versus learned occupancy and 76% versus complete rollouts at the same charged transition budget. In frozen language-model review workflows, anchored TIS beats an equally regularized mean-influence blend in 23 of 24 MMLU-Pro settings and reaches 2.4-3.4× lower MSE than rollouts on six-call FinQA reviews.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

24 of 41 references.

References · 24

  1. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  2. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  3. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  4. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  5. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 310
  6. 2021Active Testing: Sample-Efficient Model Evaluationcited by 82
  7. 2015Policy Gradient for Coherent Risk Measurescited by 76
  8. 2018An Analysis of Categorical Distributional Reinforcement Learningcited by 68
  9. 2021Universal Off-Policy Evaluationcited by 43
  10. 2024Active Evaluation Acquisition for Efficient LLM Benchmarkingcited by 35
  11. 2023Distributional Offline Policy Evaluation with Predictive Error Guaranteescited by 20
  12. 2022ReVar: Strengthening Policy Evaluation via Reduced Variance Samplingcited by 16
12 more
  1. 2023Clip-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experimentscited by 16
  2. 2018Active Testing: An Efficient and Robust Framework for Estimating Accuracycited by 15
  3. 2019Uncertainty Quantification and Exploration for Reinforcement Learningcited by 11
  4. 2023Estimation and Inference in Distributional Reinforcement Learningcited by 8
  5. 2024Statistical Efficiency of Distributional Temporal Difference Learning and Freedman's Inequality in Hilbert Spacescited by 8
  6. 2024Near-Minimax-Optimal Distributional Reinforcement Learning with a Generative Modelcited by 7
  7. 2025Near-Optimal Sample Complexity for Iterated CVaR Reinforcement Learning with a Generative Modelcited by 5
  8. 2026Logging Policy Design for Off-Policy Evaluationcited by 4
  9. 2024Distributional Off-policy Evaluation with Bellman Residual Minimizationcited by 3
  10. 2024SaVeR: Optimal Data Collection Strategy for Safe Policy Evaluation in Tabular MDPcited by 3
  11. 2026Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learningcited by 3
  12. 2026Online Inference in Distributional Temporal-Difference Learningcited by 1

Built on by

Not cited yet.