mlsanityData to 7 Oct 2026

Great Models Think Alike: Improving Model Reliability via Inter-Model Latent Agreement

arXiv 2305.01481 ↗First posted 2 May 2023cs.LG cs.AI cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Reliable application of machine learning is of primary importance to the practical deployment of deep learning methods. A fundamental challenge is that models are often unreliable due to overconfidence. In this paper, we estimate a model's reliability by measuring the agreement between its latent space, and the latent space of a foundation model. However, it is challenging to measure the agreement between two different latent spaces due to their incoherence, eg, arbitrary rotations and different dimensionality. To overcome this incoherence issue, we design a neighborhood agreement measure between latent spaces and find that this agreement is surprisingly well-correlated with the reliability of a model's predictions. Further, we show that fusing neighborhood agreement into a model's predictive confidence in a post-hoc way significantly improves its reliability. Theoretical analysis and extensive experiments on failure detection across various datasets verify the effectiveness of our method on both in-distribution and out-of-distribution settings.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 2 May 2023.

Builds on

35 of 44 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  9. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
23 more
  1. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  6. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  7. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  8. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  9. 2018Do Better ImageNet Models Transfer Better?cited by 560
  10. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  11. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  12. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  13. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  14. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  15. 2018To Trust Or Not To Trust A Classifiercited by 205
  16. 2018Do CIFAR-10 Classifiers Generalize to CIFAR-10?cited by 197
  17. 2022A Call to Reflect on Evaluation Practices for Failure Detection in Image Classificationcited by 51
  18. 2023What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifierscited by 35
  19. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  20. 2020Which Model to Transfer? Finding the Needle in the Growing Haystackcited by 21
  21. 2022Birds of a Feather Trust Together: Knowing When to Trust a Classifier via Adaptive Neighborhood Aggregationcited by 7
  22. 2023Trust, but Verify: Using Self-Supervised Probing to Improve Trustworthinesscited by 4
  23. 2022Which models are innately best at uncertainty estimation?cited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 6; 2026: 1.

10 papers.

  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  2. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 105
  3. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 74
  4. 2025SteerConf: Steering LLMs for Confidence Elicitationcited by 20
  5. 2025Enhancing Multi-Agent Debate System Performance via Confidence Expressioncited by 18
  6. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  7. 2024DECIDER: Leveraging Foundation Model Priors for Improved Model Failure Detection and Explanationcited by 6
  8. 2025Taming Object Hallucinations with Verified Atomic Confidence Estimationcited by 4
  9. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 3
  10. 2026Claim-Level Confidence Calibration for Reliable Decision Making with Large Language Modelscited by 0