mlsanityData to 13 Oct 2026

A Guide to Failure in Machine Learning: Reliability and Robustness from Foundations to Practice

arXiv 2503.00563 ↗First posted 1 Mar 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

One of the main barriers to adoption of Machine Learning (ML) is that ML models can fail unexpectedly. In this work, we aim to provide practitioners a guide to better understand why ML models fail and equip them with techniques they can use to reason about failure. Specifically, we discuss failure as either being caused by lack of reliability or lack of robustness. Differentiating the causes of failure in this way allows us to formally define why models fail from first principles and tie these definitions to engineering concepts and real-world deployment settings. Throughout the document we provide 1) a summary of important theoretic concepts in reliability and robustness, 2) a sampling current techniques that practitioners can utilize to reason about ML model reliability and robustness, and 3) examples that show how these concepts and techniques can apply to real-world settings.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 88 of its 184 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

88 of 184 references.

References · 88

  1. 2024The Llama 3 Herd of Modelscited by 14,486
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,857
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  5. 2013Intriguing properties of neural networkscited by 5,137
  6. 2017On Calibration of Modern Neural Networkscited by 4,902
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,840
  8. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,430
  9. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,902
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,992
  11. 2024SAM 2: Segment Anything in Images and Videoscited by 2,820
  12. 2017Billion-scale similarity search with GPUscited by 2,490
38 more
  1. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,371
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,340
  3. 2019Invariant Risk Minimizationcited by 1,842
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,511
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,475
  6. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,371
  7. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,362
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,327
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,297
  10. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  11. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 910
  12. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
  13. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  14. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 732
  15. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 598
  16. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 557
  17. 2020A Survey of Deep Active Learningcited by 550
  18. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  19. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  20. 2019Measuring Calibration in Deep Learningcited by 443
  21. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 442
  22. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  23. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  24. 2021Revisiting the Calibration of Modern Neural Networkscited by 388
  25. 2018Detecting and Correcting for Label Shift with Black Box Predictorscited by 365
  26. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  27. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 361
  28. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  29. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 336
  30. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 306
  31. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  32. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 277
  33. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 249
  34. 2020A Survey on the Explainability of Supervised Machine Learningcited by 225
  35. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  36. 2020Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifierscited by 185
  37. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 142
  38. 2020Physically Realizable Adversarial Examples for LiDAR Object Detectioncited by 137

50 most cited of 88 All 88 (JSON)

Built on by

Not cited yet.