mlsanityData to 7 Oct 2026

Predictive Multiplicity in Classification

arXiv 1909.06677 ↗First posted 14 Sep 2019cs.LG cs.CY stat.MLcited by 162

Not on the map. Why?

Abstract (arXiv)

Prediction problems often admit competing models that perform almost equally well. This effect challenges key assumptions in machine learning when competing models assign conflicting predictions. In this paper, we define predictive multiplicity as the ability of a prediction problem to admit competing models with conflicting predictions. We introduce formal measures to evaluate the severity of predictive multiplicity and develop integer programming tools to compute them exactly for linear classification problems. We apply our tools to measure predictive multiplicity in recidivism prediction problems. Our results show that real-world datasets may admit competing models that assign wildly conflicting predictions, and motivate the need to measure and report predictive multiplicity in model development.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 20; 2023: 19; 2024: 24; 2025: 38; 2026: 46.

162 papers.

Sort:

By area:

  1. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  2. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  3. 2021Counterfactual Explanations Can Be Manipulatedcited by 113
  4. 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 89
  5. 2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 80
  6. 2021Characterizing Fairness Over the Set of Good Models Under Selective Labelscited by 65
  7. 2021Directional Bias Amplificationcited by 54
  8. 2024Amazing Things Come From Having Many Good Modelscited by 52
  9. 2022Robust Counterfactual Explanations for Tree-Based Ensemblescited by 49
  10. 2022Rashomon Capacity: A Metric for Predictive Multiplicity in Classificationcited by 46
  11. 2025What Has a Foundation Model Found? Using Inductive Bias to Probe for World Modelscited by 46
  12. 2023Robust Counterfactual Explanations for Neural Networks With Probabilistic Guaranteescited by 45
38 more
  1. 2022Predictive Multiplicity in Probabilistic Classificationcited by 44
  2. 2021Consistent Counterfactuals for Deep Modelscited by 37
  3. 2023A Path to Simpler Models Starts With Noisecited by 33
  4. 2023Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classificationcited by 31
  5. 2023Exploring and Interacting with the Set of Good Sparse Generalized Additive Modelscited by 30
  6. 2023The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importancecited by 29
  7. 2024Robust Counterfactual Explanations in Machine Learning: A Surveycited by 29
  8. 2022Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learningcited by 26
  9. 2023An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learningcited by 25
  10. 2021Selective Ensembles for Consistent Predictionscited by 23
  11. 2021Characterizing the risk of fairwashingcited by 21
  12. 2022Rank List Sensitivity of Recommender Systems to Interaction Perturbationscited by 20
  13. 2022Improvement-Focused Causal Recourse (ICR)cited by 20
  14. 2022Toward Supporting Perceptual Complementarity in Human-AI Collaboration via Reflection on Unobservablescited by 19
  15. 2023Arbitrary Decisions are a Hidden Cost of Differentially Private Trainingcited by 19
  16. 2021Loss-Aversively Fair Classificationcited by 18
  17. 2022Algorithmic Fairness in Business Analytics: Directions for Research and Practicecited by 18
  18. 2022Learning When to Advise Human Decision Makerscited by 18
  19. 2024Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimationcited by 18
  20. 2023The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictionscited by 17
  21. 2025Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learningcited by 17
  22. 2023A Critical Survey on Fairness Benefits of Explainable AIcited by 16
  23. 2023Recourse under Model Multiplicity via Argumentative Ensembling (Technical Report)cited by 15
  24. 2024Predictive Churn with the Set of Good Modelscited by 15
  25. 2022Reconciling Individual Probability Forecastscited by 14
  26. 2021Explainability Is in the Mind of the Beholder: Establishing the Foundations of Explainable Artificial Intelligencecited by 13
  27. 2023REFORMS: Reporting Standards for Machine Learning Based Sciencecited by 13
  28. 2025Rashomon Sets for Prototypical-Part Networks: Editing Interpretable Models in Real-Timecited by 13
  29. 2021Partial Order in Chaos: Consensus on Feature Attributions in the Rashomon Setcited by 12
  30. 2022Computing the Collection of Good Models for Rule Listscited by 12
  31. 2024Algorithmic Arbitrariness in Content Moderationcited by 12
  32. 2023Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraintscited by 11
  33. 2024An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classificationcited by 11
  34. 2024Efficient Exploration of the Rashomon Set of Rule Set Modelscited by 10
  35. 2025Be Intentional About Fairness!: Fairness, Size, and Multiplicity in the Rashomon Setcited by 10
  36. 2024Predictive Multiplicity of Knowledge Graph Embeddings in Link Predictioncited by 9
  37. 2022Cross-model Fairness: Empirical Study of Fairness and Ethics Under Model Multiplicitycited by 8
  38. 2024Feature Responsiveness Scores: Model-Agnostic Explanations for Recoursecited by 8

50 most cited of 162 All 162 (JSON)