Reweighting Improves Conditional Risk Bounds
Not on the map. Why?
Abstract (arXiv)
In this work, we study the weighted empirical risk minimization (weighted ERM) schema, in which an additional data-dependent weight function is incorporated when the empirical risk function is being minimized. We show that under a general ``balanceable" Bernstein condition, one can design a weighted ERM estimator to achieve superior performance in certain sub-regions over the one obtained from standard ERM, and the superiority manifests itself through a data-dependent constant term in the error bound. These sub-regions correspond to large-margin ones in classification settings and low-variance ones in heteroscedastic regression settings, respectively. Our findings are supported by evidence from synthetic data experiments.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 24 of its 66 references that are in the map. First posted on arXiv 4 Jan 2025.
Builds on
24 of 66 references.
References · 24
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
- 2020Consistent Estimators for Learning to Defer to an Expertcited by 262
- 2016Variance-based regularization with convex objectivescited by 153
- 2022On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networkscited by 119
- 2019Towards Optimal Off-Policy Evaluation for Reinforcement Learning with Marginalized Importance Samplingcited by 118
- 2019Reliable training and estimation of variance networkscited by 63
- 2021Optimal strategies for reject option classifierscited by 61
- 2019Distribution-Independent PAC Learning of Halfspaces with Massart Noisecited by 54
- 2021Stability and Deviation Optimal Risk Bounds with Convergence Rate O(1/n)cited by 42
- 2015Learning with Square Loss: Localization through Offset Rademacher Complexitycited by 37
12 more
- 2021Variance-Aware Off-Policy Evaluation with Linear Function Approximationcited by 34
- 2021Selective Regression Under Fairness Criteriacited by 34
- 2022Understanding Why Generalized Reweighting Does Not Improve Over ERMcited by 29
- 2021Exponential Savings in Agnostic Active Learning through Abstentioncited by 23
- 2019Fast classification rates without standard margin assumptionscited by 16
- 2023Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shiftcited by 14
- 2014Learning without Concentration for General Loss Functionscited by 13
- 2020Towards Optimal Problem Dependent Generalization Error Bounds in Statistical Learning Theorycited by 13
- 2021ReLU Regression with Massart Noisecited by 10
- 2022Exponential Tail Local Rademacher Complexity Risk Bounds Without the Bernstein Conditioncited by 9
- 2022Clustering with Queries under Semi-Random Noisecited by 8
- 2023Learning to Abstain From Uninformative Datacited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.