mlsanityData to 7 Oct 2026

Beyond Top-Class Agreement: Using Divergences to Forecast Performance under Distribution Shift

arXiv 2312.08033 ↗First posted 13 Dec 2023cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Knowing if a model will generalize to data 'in the wild' is crucial for safe deployment. To this end, we study model disagreement notions that consider the full predictive distribution - specifically disagreement based on Hellinger distance, Jensen-Shannon and Kullback-Leibler divergence. We find that divergence-based scores provide better test error estimates and detection rates on out-of-distribution data compared to their top-1 counterparts. Experiments involve standard vision and foundation models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

18 of 22 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  8. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  9. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  10. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  11. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  12. 2021Assessing Generalization of SGD via Disagreementcited by 116
6 more
  1. 2022Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shiftcited by 94
  2. 2019Diversity with Cooperation: Ensemble Methods for Few-Shot Classificationcited by 70
  3. 2020Distributional Generalization: A New Kind of Generalizationcited by 41
  4. 2022A Note on "Assessing Generalization of SGD via Disagreement"cited by 16
  5. 2023(Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancycited by 16
  6. 2023Predicting Out-of-Distribution Error with Confidence Optimal Transportcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Uncertainty quantification for trustworthy deep learning: Methods and measurescited by 0