mlsanityData to 7 Oct 2026

On the Importance of Feature Separability in Predicting Out-Of-Distribution Error

arXiv 2303.15488 ↗First posted 27 Mar 2023cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Estimating the generalization performance is practically challenging on out-of-distribution (OOD) data without ground-truth labels. While previous methods emphasize the connection between distribution difference and OOD accuracy, we show that a large domain gap not necessarily leads to a low test accuracy. In this paper, we investigate this problem from the perspective of feature separability empirically and theoretically. Specifically, we propose a dataset-level score based upon feature dispersion to estimate the test accuracy under distribution shift. Our method is inspired by desirable properties of features in representation learning: high inter-class dispersion and high intra-class compactness. Our analysis shows that inter-class dispersion is strongly correlated with the model accuracy, while intra-class compactness does not reflect the generalization performance on OOD data. Extensive experiments demonstrate the superiority of our method in both prediction performance and computational efficiency.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

30 of 47 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  4. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  5. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  6. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  7. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  8. 2017Exploring Generalization in Deep Learningcited by 595
  9. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  10. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  11. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  12. 2017Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learningcited by 192
18 more
  1. 2016Distributionally Robust Stochastic Optimization with Wasserstein Distancecited by 182
  2. 2021Learning to Diversify for Single Domain Generalizationcited by 170
  3. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  4. 2022How to Exploit Hyperspherical Embeddings for Out-of-Distribution Detection?cited by 128
  5. 2021Assessing Generalization of SGD via Disagreementcited by 116
  6. 2021Predicting with Confidence on Unseen Distributionscited by 116
  7. 2018Predicting the Generalization Gap in Deep Networks with Margin Distributionscited by 111
  8. 2020Predicting Neural Network Accuracy from Weightscited by 107
  9. 2020Are Labels Always Necessary for Classifier Accuracy Evaluation?cited by 104
  10. 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
  11. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  12. 2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
  13. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 57
  14. 2020Computing the Testing Error without a Testing Setcited by 54
  15. 2022Predicting Out-of-Distribution Error with the Projection Normcited by 50
  16. 2019Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networkscited by 43
  17. 2017Estimating Accuracy from Unlabeled Data: A Probabilistic Logic Approachcited by 27
  18. 2019Towards Task and Architecture-Independent Generalization Gap Predictorscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 6; 2026: 1.

15 papers.

Sort:

By area:

  1. 2024Energy-based Automated Model Evaluationcited by 16
  2. 2024MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shiftscited by 8
  3. 2024What Does Softmax Probability Tell Us about Classifiers Ranking Across Diverse Test Conditions?cited by 6
  4. 2025Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settingscited by 4
  5. 2025A Geometry-Based View of Mahalanobis OOD Detectioncited by 4
  6. 2024Leveraging Gradients for Unsupervised Accuracy Estimation under Distribution Shiftcited by 3
  7. 2025FedPCA: Noise-Robust Fair Federated Learning via Performance-Capacity Analysiscited by 2
  8. 2024Source-Free Domain-Invariant Performance Predictioncited by 1
  9. 2024Poor-Supervised Evaluation for SuperLLM via Mutual Consistencycited by 1
  10. 2024Learning to Rank Pre-trained Vision-Language Models for Downstream Taskscited by 1
  11. 2025Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Rankingcited by 1
  12. 2025ODP-Bench: Benchmarking Out-of-Distribution Performance Predictioncited by 1
3 more
  1. 2026Bridging Domain Expertise and Generalization for Performance Estimationcited by 1
  2. 2024Assessing Model Generalization in Vicinitycited by 0
  3. 2025A Guide to Failure in Machine Learning: Reliability and Robustness from Foundations to Practicecited by 0