mlsanityData to 7 Oct 2026

Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning

arXiv 2207.02598 ↗First posted 6 Jul 2022cs.LG cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) models are typically optimized for their accuracy on a given dataset. However, this predictive criterion rarely captures all desirable properties of a model, in particular how well it matches a domain expert's understanding of a task. Underspecification refers to the existence of multiple models that are indistinguishable in their in-domain accuracy, even though they differ in other desirable properties such as out-of-distribution (OOD) performance. Identifying these situations is critical for assessing the reliability of ML models. We formalize the concept of underspecification and propose a method to identify and partially address it. We train multiple models with an independence constraint that forces them to implement different functions. They discover predictive features that are otherwise ignored by standard empirical risk minimization (ERM), which we then distill into a global model with superior OOD performance. Importantly, we constrain the models to align with the data manifold to ensure that they discover meaningful features. We demonstrate the method on multiple datasets in computer vision (collages, WILDS-Camelyon17, GQA) and discuss general implications of underspecification. Most notably, in-domain performance cannot serve for OOD model selection without additional assumptions.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 57 of its 85 references that are in the map. First posted on arXiv 6 Jul 2022.

Builds on

57 of 85 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  3. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  4. 2019Invariant Risk Minimizationcited by 1,832
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  7. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  8. 2020In Search of Lost Domain Generalizationcited by 1,042
  9. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  10. 2015Causal inference using invariant prediction: identification and confidence intervalscited by 533
  11. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 494
  12. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
38 more
  1. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  2. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  3. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  4. 2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
  5. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  6. 2019Counterfactual Visual Explanationscited by 312
  7. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  8. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  9. 2020Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learningcited by 292
  10. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  11. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  12. 2020What Should Not Be Contrastive in Contrastive Learningcited by 224
  13. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  14. 2019Predictive Multiplicity in Classificationcited by 162
  15. 2021Reading Race: AI Recognises Patient's Racial Identity In Medical Imagescited by 155
  16. 2016Correlation Alignment for Unsupervised Domain Adaptationcited by 146
  17. 2021On Calibration and Out-of-domain Generalizationcited by 134
  18. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  19. 2019Robust Learning with Jacobian Regularizationcited by 127
  20. 2021Scalable Marginal Likelihood Estimation for Model Selection in Deep Learningcited by 105
  21. 2020On Disentangled Representations Learned From Correlated Datacited by 94
  22. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  23. 2017Tips and Tricks for Visual Question Answering: Learnings from the 2017 Challengecited by 76
  24. 2020Hidden Markov Nonlinear ICA: Unsupervised Learning from Nonstationary Time Seriescited by 76
  25. 2020On the Value of Out-of-Distribution Testing: An Example of Goodhart's Lawcited by 74
  26. 2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
  27. 2020Roses Are Red, Violets Are Blue... but Should Vqa Expect Them To?cited by 69
  28. 2022Agree to Disagree: Diversity through Disagreement for Better Transferabilitycited by 69
  29. 2022Domain-Adjusted Regression or: ERM May Already Learn Features Sufficient for Out-of-Distribution Generalizationcited by 69
  30. 2020Learning What Makes a Difference from Counterfactual Examples and Gradient Supervisioncited by 66
  31. 2017Invariant Causal Prediction for Sequential Datacited by 63
  32. 2021What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?cited by 63
  33. 2022Diversify and Disambiguate: Learning From Underspecified Datacited by 55
  34. 2020Evaluating Models' Local Decision Boundaries via Contrast Setscited by 24
  35. 2019Ensembles of Locally Independent Prediction Modelscited by 22
  36. 2020Ridge Rider: Finding Diverse Solutions by Following Eigenvectors of the Hessiancited by 20
  37. 2021RATT: Leveraging Unlabeled Data to Guarantee Generalizationcited by 20
  38. 2021Designing Counterfactual Generators using Deep Model Inversioncited by 15

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 6; 2025: 5; 2026: 4.

26 papers.

Sort:

  1. 2022Mechanistic Mode Connectivitycited by 50
  2. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  3. 2024Neural Redshift: Random Networks are not Random Functionscited by 43
  4. 2023A Path to Simpler Models Starts With Noisecited by 33
  5. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?cited by 32
  6. 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performancecited by 25
  7. 2023A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepediacited by 24
  8. 2024Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimationcited by 18
  9. 2023The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictionscited by 17
  10. 2025Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wildcited by 13
  11. 2025The Dead Salmons of AI Interpretabilitycited by 10
  12. 2023Learning Diverse Features in Vision Transformers for Improved Generalizationcited by 4
14 more
  1. 2023Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Taskscited by 4
  2. 2023Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensemblescited by 4
  3. 2023Unraveling the Key Components of OOD Generalization via Diversificationcited by 4
  4. 2023Can we Agree? On the Rash=omon Effect and the Reliability of Post-Hoc Explainable AIcited by 2
  5. 2024Scalable Ensemble Diversification for OOD Generalization and Detectioncited by 2
  6. 2024OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?cited by 2
  7. 2026Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Taskscited by 2
  8. 2025Meta-Statistical Learning: Supervised Learning of Statistical Estimatorscited by 1
  9. 2025On Aligning Prediction Models with Clinical Experiential Learning: A Prostate Cancer Case Studycited by 1
  10. 2024Learning from Uncertain Data: From Possible Worlds to Possible Modelscited by 0
  11. 2024Hallucination Detection in Virtually-Stained Histology: A Latent Space Baselinecited by 0
  12. 2026What Matters to an LLM? Behavioral and Computational Evidences from Summarizationcited by 0
  13. 2026The limits of interpretability in multiple linear regressioncited by 0
  14. 2026Validating Causal Abstraction Metrics on Simulated Complex Systemscited by 0