# Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning

arXiv: https://arxiv.org/abs/2207.02598 · HTML: https://mlsanity.com/paper/2207.02598/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 85 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 30 of 57)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [The Pitfalls of Simplicity Bias in Neural Networks](https://mlsanity.com/paper/2006.07710/) (2020)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Counterfactual Visual Explanations](https://mlsanity.com/paper/1904.07451/) (2019)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning](https://mlsanity.com/paper/2012.09816/) (2020)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [What Should Not Be Contrastive in Contrastive Learning](https://mlsanity.com/paper/2008.05659/) (2020)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [Correlation Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1612.01939/) (2016)
- [On Calibration and Out-of-domain Generalization](https://mlsanity.com/paper/2102.10395/) (2021)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)

## Built on by (most cited first, 26 of 26)

- [Mechanistic Mode Connectivity](https://mlsanity.com/paper/2211.08422/) (2022)
- [ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets](https://mlsanity.com/paper/2209.00613/) (2022)
- [Neural Redshift: Random Networks are not Random Functions](https://mlsanity.com/paper/2403.02241/) (2024)
- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [A Path to Simpler Models Starts With Noise](https://mlsanity.com/paper/2310.19726/) (2023)
- [It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance](https://mlsanity.com/paper/2305.09022/) (2023)
- [A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia](https://mlsanity.com/paper/2312.02073/) (2023)
- [Dropout-Based Rashomon Set Exploration for Efficient Predictive Multiplicity Estimation](https://mlsanity.com/paper/2402.00728/) (2024)
- [The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictions](https://mlsanity.com/paper/2304.10655/) (2023)
- [Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild](https://mlsanity.com/paper/2503.10065/) (2025)
- [The Dead Salmons of AI Interpretability](https://mlsanity.com/paper/2512.18792/) (2025)
- [Learning Diverse Features in Vision Transformers for Improved Generalization](https://mlsanity.com/paper/2308.16274/) (2023)
- [Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks](https://mlsanity.com/paper/2310.02230/) (2023)
- [Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensembles](https://mlsanity.com/paper/2311.16176/) (2023)
- [Unraveling the Key Components of OOD Generalization via Diversification](https://mlsanity.com/paper/2312.16313/) (2023)
- [Meta-Statistical Learning: Supervised Learning of Statistical Estimators](https://mlsanity.com/paper/2502.12088/) (2025)
- [Can we Agree? On the Rash=omon Effect and the Reliability of Post-Hoc Explainable AI](https://mlsanity.com/paper/2308.07247/) (2023)
- [Scalable Ensemble Diversification for OOD Generalization and Detection](https://mlsanity.com/paper/2409.16797/) (2024)
- [OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?](https://mlsanity.com/paper/2410.02735/) (2024)
- [Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks](https://mlsanity.com/paper/2607.17624/) (2026)
- [On Aligning Prediction Models with Clinical Experiential Learning: A Prostate Cancer Case Study](https://mlsanity.com/paper/2509.04053/) (2025)
- [Learning from Uncertain Data: From Possible Worlds to Possible Models](https://mlsanity.com/paper/2405.18549/) (2024)
- [Hallucination Detection in Virtually-Stained Histology: A Latent Space Baseline](https://mlsanity.com/paper/2411.15060/) (2024)
- [What Matters to an LLM? Behavioral and Computational Evidences from Summarization](https://mlsanity.com/paper/2602.00459/) (2026)
- [The limits of interpretability in multiple linear regression](https://mlsanity.com/paper/2606.16013/) (2026)
- [Validating Causal Abstraction Metrics on Simulated Complex Systems](https://mlsanity.com/paper/2607.00267/) (2026)

## Abstract (arXiv)

Machine learning (ML) models are typically optimized for their accuracy on a given dataset. However, this predictive criterion rarely captures all desirable properties of a model, in particular how well it matches a domain expert's understanding of a task. Underspecification refers to the existence of multiple models that are indistinguishable in their in-domain accuracy, even though they differ in other desirable properties such as out-of-distribution (OOD) performance. Identifying these situations is critical for assessing the reliability of ML models. We formalize the concept of underspecification and propose a method to identify and partially address it. We train multiple models with an independence constraint that forces them to implement different functions. They discover predictive features that are otherwise ignored by standard empirical risk minimization (ERM), which we then distill into a global model with superior OOD performance. Importantly, we constrain the models to align with the data manifold to ensure that they discover meaningful features. We demonstrate the method on multiple datasets in computer vision (collages, WILDS-Camelyon17, GQA) and discuss general implications of underspecification. Most notably, in-domain performance cannot serve for OOD model selection without additional assumptions.