Monotonic Risk Relationships under Distribution Shifts for Regularized Risk Minimization
Not on the map. Why?
Abstract (arXiv)
Machine learning systems are often applied to data that is drawn from a different distribution than the training distribution. Recent work has shown that for a variety of classification and signal reconstruction problems, the out-of-distribution performance is strongly linearly correlated with the in-distribution performance. If this relationship or more generally a monotonic one holds, it has important consequences. For example, it allows to optimize performance on one distribution as a proxy for performance on the other. In this paper, we study conditions under which a monotonic relationship between the performances of a model on two distributions is expected. We prove an exact asymptotic linear relation for squared error and a monotonic relation for misclassification error for ridge-regularized general linear models under covariate shift, as well as an approximate linear relation for linear inverse problems.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 20 of its 42 references that are in the map. First posted on arXiv 20 Oct 2022.
Builds on
20 of 42 references.
References · 20
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2017Mask R-CNNcited by 7,277
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2015SSD: Single Shot MultiBox Detectorcited by 3,130
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
- 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
- 2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
- 2020The Effect of Natural Distribution Shift on Question Answering Modelscited by 104
8 more
- 2019BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performancecited by 89
- 2019Cold Case: The Lost MNIST Digitscited by 62
- 2021Measuring Robustness in Deep Learning Based Compressive Sensingcited by 53
- 2021Pseudo-labeling for Scalable 3D Object Detectioncited by 32
- 2020Generalization Error of Generalized Linear Models in High Dimensionscited by 25
- 2021Covariate Shift in High-Dimensional Random Feature Regressioncited by 24
- 2020Why do classifier accuracies show linear trends under distribution shift?cited by 21
- 2013Robust Subspace Clustering via Thresholdingcited by 16
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 1.
2 papers.