mlsanityData to 7 Oct 2026

Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis

arXiv 1606.04316 ↗First posted 14 Jun 2016stat.ML cs.LGcited by 92

Not on the map. Why?

Abstract (arXiv)

The machine learning community adopted the use of null hypothesis significance testing (NHST) in order to ensure the statistical validity of results. Many scientific fields however realized the shortcomings of frequentist reasoning and in the most radical cases even banned its use in publications. We should do the same: just as we have embraced the Bayesian paradigm in the development of new machine learning methods, so we should also use it in the analysis of our own results. We argue for abandonment of NHST by exposing its fallacies and, more importantly, offer better - more sound and useful - alternatives for it.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jun 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 24; 2023: 11; 2024: 15; 2025: 13; 2026: 12.

92 papers.

Sort:

By area:

  1. 2021DeepSMOTE: Fusing Deep Learning and SMOTE for Imbalanced Datacited by 62
  2. 2022MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Predictioncited by 59
  3. 2021How to avoid machine learning pitfalls: a guide for academic researcherscited by 34
  4. 2022deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networkscited by 28
  5. 2023An Approach to Multiple Comparison Benchmark Evaluations that is Stable Under Manipulation of the Comparate Setcited by 27
  6. 2021Zero-Shot Scene Graph Relation Prediction through Commonsense Knowledge Integrationcited by 21
  7. 2022Statistical Comparisons of Classifiers by Generalized Stochastic Dominancecited by 20
  8. 2021STUDD: A Student-Teacher Method for Unsupervised Concept Drift Detectioncited by 18
  9. 2025Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learningcited by 15
  10. 2021Concept Drift Detection from Multi-Class Imbalanced Data Streamscited by 13
  11. 2021Embedding Graph Convolutional Networks in Recurrent Neural Networks for Predictive Monitoringcited by 12
  12. 2024Confidence intervals uncovered: Are we ready for real-world medical imaging AI?cited by 12
38 more
  1. 2021A Comparative Evaluation of Quantification Methodscited by 11
  2. 2021On the validity of pre-trained transformers for natural language processing in the software engineering domaincited by 11
  3. 2024Statistical Multicriteria Benchmarking via the GSD-Frontcited by 11
  4. 2025LLM Embeddings for Deep Learning on Tabular Datacited by 11
  5. 2025Is GPT-OSS Good? A Comprehensive Evaluation of OpenAI's Latest Open Source Modelscited by 11
  6. 2022TASKED: Transformer-based Adversarial learning for human activity recognition using wearable sensors via Self-KnowledgE Distillationcited by 10
  7. 2025Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarkscited by 10
  8. 2022Experimental Standards for Deep Learning in Natural Language Processing Researchcited by 9
  9. 2022A Bayesian Bradley-Terry model to compare multiple ML algorithms on multiple data setscited by 9
  10. 2024Towards Better Open-Ended Text Generation: A Multicriteria Evaluation Frameworkcited by 9
  11. 2025Statistical Multicriteria Evaluation of LLM-Generated Textcited by 9
  12. 2023Kernel Density Estimation for Multiclass Quantificationcited by 8
  13. 2022Model Optimization in Imbalanced Regressioncited by 7
  14. 2023Semi-Supervised Constrained Clustering: An In-Depth Overview, Ranked Taxonomy and Future Research Directionscited by 7
  15. 2024When is an Embedding Model More Promising than Another?cited by 7
  16. 2021Model-based metrics: Sample-efficient estimates of predictive model subpopulation performancecited by 6
  17. 2021Retweet communities reveal the main sources of hate speechcited by 5
  18. 2021Test for non-negligible adverse shiftscited by 5
  19. 2022Towards a Data Privacy-Predictive Performance Trade-offcited by 5
  20. 2022Handling Imbalanced Classification Problems With Support Vector Machines via Evolutionary Bilevel Optimizationcited by 5
  21. 2022Meta-Learning for Unsupervised Outlier Detection with Optimal Transportcited by 5
  22. 2024A Tutorial on the Design, Experimentation and Application of Metaheuristic Algorithms to Real-World Optimization Problemscited by 5
  23. 2022Differentially-Private Data Synthetisation for Efficient Re-Identification Risk Controlcited by 4
  24. 2025SortBench: Benchmarking LLMs based on their ability to sort listscited by 4
  25. 2025Fusion or Confusion? Multimodal Complexity Is Not All You Needcited by 4
  26. 2021Learning Abstract Task Representationscited by 3
  27. 2022Statistical Model Criticism of Variational Auto-Encoderscited by 3
  28. 2022Differential testing for machine learning: an analysis for classification algorithms beyond deep learningcited by 3
  29. 2022Accelerated and interpretable oblique random survival forestscited by 3
  30. 2023D3A-TS: Denoising-Driven Data Augmentation in Time Seriescited by 3
  31. 2024AnyLoss: Transforming Classification Metrics into Loss Functionscited by 3
  32. 2021Unsupervised Feature Ranking via Attribute Networkscited by 2
  33. 2022Comparing Two Samples Through Stochastic Dominance: A Graphical Approachcited by 2
  34. 2024Time Series Data Augmentation as an Imbalanced Learning Problemcited by 2
  35. 2024A State-of-the-Art Morphosyntactic Parser and Lemmatizer for Ancient Greekcited by 2
  36. 2021ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddingscited by 1
  37. 2021Model Compression for Dynamic Forecast Combinationcited by 1
  38. 2022Separating Rule Discovery and Global Solution Composition in a Learning Classifier Systemcited by 1

50 most cited of 92 All 92 (JSON)