Data to 7 Oct 2026

Underspecification Presents Challenges for Credibility in Modern Machine Learning

arXiv 2011.03395 ↗First posted 6 Nov 2020cs.LG stat.MLcited by 488

Not on the map. Why?

Abstract (arXiv)

ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our results show the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployment in any domain.

Key facts. Cited by 488 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 102; 2022: 99; 2023: 77; 2024: 59; 2025: 45; 2026: 106.

488 papers.

Sort:

By area:

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
2021Robust fine-tuning of zero-shot modelscited by 785
2021Interpretable Machine Learning: Fundamental Principles and 10 Grand Challengescited by 364
2023TRAK: Attributing Model Behavior at Scalecited by 273
2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
2023Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hackingcited by 195
2022Metrics reloaded: Recommendations for image analysis validationcited by 190
2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
2022Datamodels: Predicting Predictions from Training Datacited by 186
2021Alignment of Language Agentscited by 167

38 more

2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
2021Ensemble of Averages: Improving Model Selection and Boosting Performance in Domain Generalizationcited by 144
2022Training Data Influence Analysis and Estimation: A Surveycited by 134
2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 120
2021Counterfactual Generative Networkscited by 111
2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 100
2021A Structured Dictionary Perspective on Implicit Neural Representationscited by 100
2021Do Feature Attribution Methods Correctly Attribute Features?cited by 98
2021What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methodscited by 98
2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
2022Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalizationcited by 97
2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
2023Eliciting Human Preferences with Language Modelscited by 90
2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 89
2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 89
2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
2021Learning Neural Network Subspacescited by 82
2021Size-Invariant Graph Representations for Graph Classification Extrapolationscited by 82
2021Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learningcited by 81
2022Exploring the Whole Rashomon Set of Sparse Decision Treescited by 80
2021Desiderata for Representation Learning: A Causal Perspectivecited by 76
2023Eight Things to Know about Large Language Modelscited by 73
2021Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answeringcited by 71
2022Identifiability of deep generative models without auxiliary informationcited by 70
2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
2021The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Modelscited by 63
2024Calibrating Large Language Models Using Their Generations Onlycited by 63
2021Meta-Learning to Compositionally Generalizecited by 62
2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 57
2021No One Representation to Rule Them All: Overlapping Features of Training Methodscited by 57
2022LMPriors: Pre-Trained Language Models as Task-Specific Priorscited by 56
2022PINTO: Faithful Language Reasoning Using Prompt-Generated Rationalescited by 56
2021Causally motivated Shortcut Removal Using Auxiliary Labelscited by 55

50 most cited of 488 All 488 (JSON)