IW-GAE: Importance Weighted Group Accuracy Estimation for Improved Calibration and Model Selection in Unsupervised Domain Adaptation
Not on the map. Why?
Abstract (arXiv)
Distribution shifts pose significant challenges for model calibration and model selection tasks in the unsupervised domain adaptation problem -- a scenario where the goal is to perform well in a distribution shifted domain without labels. In this work, we tackle difficulties coming from distribution shifts by developing a novel importance weighted group accuracy estimator. Specifically, we present a new perspective of addressing the model calibration and model selection tasks by estimating the group accuracy. Then, we formulate an optimization problem for finding an importance weight that leads to an accurate group accuracy estimation with theoretical analyses. Our extensive experiments show that our approach improves state-of-the-art performances by 22% in the model calibration task and 14% in the model selection task.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 70 references that are in the map. First posted on arXiv 16 Oct 2023.
Builds on
41 of 70 references.
References · 41
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017On Calibration of Modern Neural Networkscited by 4,960
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2015Domain-Adversarial Training of Neural Networkscited by 3,044
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
- 2016Concrete Problems in AI Safetycited by 1,642
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
- 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
29 more
- 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
- 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
- 2015Deep Learning and the Information Bottleneck Principlecited by 802
- 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
- 2017Maximum Classifier Discrepancy for Unsupervised Domain Adaptationcited by 683
- 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
- 2019Release Strategies and the Social Impacts of Language Modelscited by 513
- 2017VisDA: The Visual Domain Adaptation Challengecited by 435
- 2019Bridging Theory and Algorithm for Domain Adaptationcited by 332
- 2017Concrete Dropoutcited by 227
- 2020Uncertainty Quantification and Deep Ensemblescited by 139
- 2017Non-convex Optimization for Machine Learningcited by 133
- 2020Improving model calibration with accuracy versus uncertainty optimizationcited by 123
- 2021Cycle Self-Training for Domain Adaptationcited by 120
- 2019Uncertainty-guided Continual Learning with Bayesian Neural Networkscited by 102
- 2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
- 2020Self-training Avoids Using Spurious Features Under Domain Shiftcited by 69
- 2022A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibrationcited by 49
- 2021A Theory of Label Propagation for Subpopulation Shiftcited by 48
- 2021Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibrationcited by 48
- 2021PAC Prediction Sets Under Covariate Shiftcited by 45
- 2021Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Densitycited by 44
- 2020Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptationcited by 43
- 2020Transferable Calibration with Lower Bias and Variance in Domain Adaptationcited by 41
- 2022Robust Calibration with Multi-domain Temperature Scalingcited by 39
- 2020Being Bayesian about Categorical Probabilitycited by 35
- 2023Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?cited by 14
- 2013The cost of using exact confidence intervals for a binomial proportioncited by 11
- 2021Frustratingly Easy Uncertainty Estimation for Distribution Shiftcited by 1
Built on by
Citing papers per year: 2024: 1.
1 paper.