Tailoring to the Tails: Risk Measures for Fine-Grained Tail Sensitivity
Not on the map. Why?
Abstract (arXiv)
Expected risk minimization (ERM) is at the core of many machine learning systems. This means that the risk inherent in a loss distribution is summarized using a single number - its average. In this paper, we propose a general approach to construct risk measures which exhibit a desired tail sensitivity and may replace the expectation operator in ERM. Our method relies on the specification of a reference distribution with a desired tail behaviour, which is in a one-to-one correspondence to a coherent upper probability. Any risk measure, which is compatible with this upper probability, displays a tail sensitivity which is finely tuned to the reference distribution. As a concrete example, we focus on divergence risk measures based on f-divergence ambiguity sets, which are a widespread tool used to foster distributional robustness of machine learning systems. For instance, we show how ambiguity sets based on the Kullback-Leibler divergence are intricately tied to the class of subexponential random variables. We elaborate the connection of divergence risk measures and rearrangement invariant Banach norms.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 26 of its 97 references that are in the map. First posted on arXiv 5 Aug 2022.
Builds on
26 of 97 references.
References · 26
- 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
- 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
- 2016Statistics of Robust Optimization: A Generalized Empirical Likelihood Approachcited by 182
- 2016Variance-based regularization with convex objectivescited by 153
- 2013Risk-sensitive Reinforcement Learningcited by 129
- 2020The Heavy-Tail Phenomenon in SGDcited by 122
- 2020Tilted Empirical Risk Minimizationcited by 103
- 2019Fairness risk measurescited by 85
- 2019Distributionally Robust Optimization: A Reviewcited by 75
- 2015Policy Gradient for Coherent Risk Measurescited by 74
- 2020Coping with Label Shift via Distributionally Robust Optimisationcited by 69
- 2021Risk-Averse Offline Reinforcement Learningcited by 65
14 more
- 2017Learning with Average Top-k Losscited by 52
- 2013Loss minimization and parameter estimation with heavy tailscited by 50
- 2019Adaptive Sampling for Stochastic Risk-Averse Learningcited by 39
- 2014Empirical risk minimization for heavy-tailed lossescited by 37
- 2020Improving Robustness via Risk Averse Distributional Reinforcement Learningcited by 28
- 2018Subspace Embedding and Linear Regression with Orlicz Normcited by 19
- 2020Statistical Learning with Conditional Value at Riskcited by 19
- 2020Minimizing Spectral Risk Measures Applied to Markov Decision Processescited by 12
- 2019Estimation of Spectral Risk Measurescited by 9
- 2022Risk Measures and Upper Probabilities: Coherence and Stratificationcited by 9
- 2021Spectral risk-based learning using unbounded lossescited by 8
- 2019Efficient Symmetric Norm Regression via Linear Sketchingcited by 7
- 2022Supervised Learning with General Risk Functionalscited by 6
- 2019General risk measures for robust machine learningcited by 1
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.