mlsanityData to 7 Oct 2026

Estimating Generalization under Distribution Shifts via Domain-Invariant Representations

arXiv 2007.03511 ↗First posted 6 Jul 2020cs.LG stat.MLcited by 56

Not on the map. Why?

Abstract (arXiv)

When machine learning models are deployed on a test distribution different from the training distribution, they can perform poorly, but overestimate their performance. In this work, we aim to better estimate a model's performance under distribution shift, without supervision. To do so, we use a set of domain-invariant predictors as a proxy for the unknown, true target labels. Since the error of the resulting risk estimate depends on the target risk of the proxy model, we study generalization of domain-invariant representations and show that the complexity of the latent representation has a significant influence on the target risk. Empirically, our approach (1) enables self-tuning of domain adaptation models, and (2) accurately estimates the target error of given models under distribution shift. Other applications include model selection, deciding early stopping and error detection.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 10; 2023: 14; 2024: 10; 2025: 8; 2026: 4.

56 papers.

Sort:

By area:

  1. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  2. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  3. 2021Assessing Generalization of SGD via Disagreementcited by 116
  4. 2021Fair Mixup: Fairness via Interpolationcited by 109
  5. 2022Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shiftcited by 94
  6. 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
  7. 2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
  8. 2021Detecting Errors and Estimating Accuracy on Unlabeled Data with Self-training Ensemblescited by 54
  9. 2022Predicting Out-of-Distribution Error with the Projection Normcited by 50
  10. 2022Tree Mover's Distance: Bridging Graph Metrics and Stability of Graph Neural Networkscited by 39
  11. 2021Measuring Generalization with Optimal Transportcited by 24
  12. 2023Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy Estimationcited by 22
38 more
  1. 2023DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networkscited by 20
  2. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 20
  3. 2022Estimating and Explaining Model Performance When Both Covariates and Labels Shiftcited by 19
  4. 2023(Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancycited by 16
  5. 2024Energy-based Automated Model Evaluationcited by 16
  6. 2021Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Featurescited by 14
  7. 2025Handling Out-of-Distribution Data: A Surveycited by 14
  8. 2024A Framework for Efficient Model Evaluation through Stratification, Sampling, and Estimationcited by 13
  9. 2022Three New Validators and a Large-Scale Benchmark Ranking for Unsupervised Domain Adaptationcited by 12
  10. 2024AETTA: Label-Free Accuracy Estimation for Test-Time Adaptationcited by 9
  11. 2021Towards Evaluating the Robustness of Neural Networks Learned by Transductioncited by 8
  12. 2023Moderately Distributional Exploration for Domain Generalizationcited by 8
  13. 2024MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shiftscited by 8
  14. 2023ProbVLM: Probabilistic Adapter for Frozen Vision-Language Modelscited by 7
  15. 2023Selecting which Dense Retriever to use for Zero-Shot Searchcited by 7
  16. 2024Predicting the Performance of Foundation Models via Agreement-on-the-Linecited by 7
  17. 2022HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictionscited by 6
  18. 2023CAME: Contrastive Automated Model Evaluationcited by 6
  19. 2023Simple Transferability Estimation for Regression Taskscited by 5
  20. 2021DAPPER: Label-Free Performance Estimation after Personalization for Heterogeneous Mobile Sensingcited by 4
  21. 2022Predicting Out-of-Domain Generalization with Neighborhood Invariancecited by 4
  22. 2023INSURE: An Information Theory Inspired Disentanglement and Purification Model for Domain Generalizationcited by 4
  23. 2025Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settingscited by 4
  24. 2023ASPEST: Bridging the Gap Between Active Learning and Selective Predictioncited by 3
  25. 2023Unsupervised Accuracy Estimation of Deep Visual Models using Domain-Adaptive Adversarial Perturbation without Source Samplescited by 3
  26. 2023Deep Unsupervised Domain Adaptation for Time Series Classification: a Benchmarkcited by 3
  27. 2024Learning Invariant Causal Mechanism from Vision-Language Modelscited by 3
  28. 2024Unveiling AI's Blind Spots: An Oracle for In-Domain, Out-of-Domain, and Adversarial Errorscited by 3
  29. 2021Towards Adversarial Robustness via Transductive Learningcited by 2
  30. 2023Towards Regulatable AI Systems: Technical Gaps and Policy Opportunitiescited by 2
  31. 2023Distance Matters For Improving Performance Estimation Under Covariate Shiftcited by 2
  32. 2024Causal Invariance Learning via Efficient Nonconvex Optimizationcited by 2
  33. 2025Why Cannot Neural Networks Master Extrapolation? Insights from Physical Lawscited by 2
  34. 2026Distribution Shift Is Key to Learning Invariant Predictioncited by 2
  35. 2026Pitfalls of Unlabeled Disagreement-Based Drift Detection in Streaming Tree Ensemblescited by 2
  36. 2025Subgroups Matter for Robust Bias Mitigationcited by 1
  37. 2025ODD: Overlap-aware Estimation of Model Performance under Distribution Shiftcited by 1
  38. 2022Repeated Environment Inference for Invariant Learningcited by 0

50 most cited of 56 All 56 (JSON)