mlsanityData to 7 Oct 2026

Diversify and Disambiguate: Learning From Underspecified Data

arXiv 2202.03418 ↗First posted 7 Feb 2022cs.LG stat.MLcited by 55

Not on the map. Why?

Abstract (arXiv)

Many datasets are underspecified: there exist multiple equally viable solutions to a given task. Underspecification can be problematic for methods that learn a single hypothesis because different functions that achieve low training loss can focus on different predictive features and thus produce widely varying predictions on out-of-distribution data. We propose DivDis, a simple two-stage framework that first learns a diverse collection of hypotheses for a task by leveraging unlabeled data from the test distribution. We then disambiguate by selecting one of the discovered hypotheses using minimal additional supervision, in the form of additional labels or inspection of function visualization. We demonstrate the ability of DivDis to find hypotheses that use robust features in image classification and natural language processing problems with underspecification.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 52 of its 79 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

52 of 79 references.

References · 52

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  10. 2019Invariant Risk Minimizationcited by 1,832
  11. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
38 more
  1. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  2. 2017A Closer Look at Memorization in Deep Networkscited by 936
  3. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  4. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  7. 2018Disentangling by Factorisingcited by 614
  8. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  9. 2015Weight Uncertainty in Neural Networkscited by 554
  10. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  11. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 498
  12. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  13. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
  14. 2015Illuminating search spaces by mapping elitescited by 406
  15. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  16. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  17. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  18. 2020Environment Inference for Invariant Learningcited by 306
  19. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  20. 2018Implicit Bias of Gradient Descent on Linear Convolutional Networkscited by 251
  21. 2019Causality for Machine Learningcited by 242
  22. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 215
  23. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  24. 2020Large-Scale Methods for Distributionally Robust Optimizationcited by 179
  25. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  26. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  27. 2016Correlation Alignment for Unsupervised Domain Adaptationcited by 146
  28. 2020Learning from Failure: Training Debiased Classifier from Biased Classifiercited by 133
  29. 2019Learning Neural Causal Models from Unknown Interventionscited by 113
  30. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 111
  31. 2017Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agentscited by 107
  32. 2021Repulsive Deep Ensembles are Bayesiancited by 103
  33. 2021Extending the WILDS Benchmark for Unsupervised Adaptationcited by 85
  34. 2021Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalizationcited by 83
  35. 2021Learning Neural Network Subspacescited by 82
  36. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  37. 2021Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspectivecited by 56
  38. 2021DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimationcited by 48

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 21; 2024: 13; 2025: 6; 2026: 1.

55 papers.

Sort:

By area:

  1. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  2. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  3. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  4. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  5. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  6. 2022Agree to Disagree: Diversity through Disagreement for Better Transferabilitycited by 69
  7. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  8. 2022MaskTune: Mitigating Spurious Correlations by Forcing to Explorecited by 66
  9. 2023Towards Last-layer Retraining for Group Robustness with Fewer Annotationscited by 54
  10. 2022Mechanistic Mode Connectivitycited by 50
  11. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  12. 2022Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learningcited by 26
38 more
  1. 2023Simplicity Bias in 1-Hidden Layer Neural Networkscited by 26
  2. 2023Discovering environments with XRMcited by 26
  3. 2022Chroma-VAE: Mitigating Shortcut Learning with Generative Classifierscited by 17
  4. 2023Pathologies of Predictive Diversity in Deep Ensemblescited by 16
  5. 2024(Implicit) Ensembles of Ensembles: Epistemic Uncertainty Collapse in Large Modelscited by 15
  6. 2023Improving Domain Generalization with Domain Relationscited by 14
  7. 2025Safety Reasoning with Guidelinescited by 14
  8. 2024QT-DoG: Quantization-aware Training for Domain Generalizationcited by 13
  9. 2024Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlationscited by 12
  10. 2024Contextualized Evaluations: Judging Language Model Responses to Underspecified Queriescited by 12
  11. 2022OccamNets: Mitigating Dataset Bias by Favoring Simpler Hypothesescited by 11
  12. 2024Test-Time Alignment via Hypothesis Reweightingcited by 11
  13. 2023Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shiftscited by 9
  14. 2023Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Featurescited by 9
  15. 2024Clarify: Improving Model Robustness With Natural Language Correctionscited by 8
  16. 2023Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustnesscited by 7
  17. 2023Bias Amplification Enhances Minority Group Performancecited by 6
  18. 2023Controlling Learned Effects to Reduce Spurious Correlations in Text Classifierscited by 5
  19. 2023Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shiftscited by 5
  20. 2024The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlationscited by 5
  21. 2023Learning Diverse Features in Vision Transformers for Improved Generalizationcited by 4
  22. 2023Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Taskscited by 4
  23. 2023Diversify & Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreementcited by 4
  24. 2023Mitigating Shortcut Learning with Diffusion Counterfactuals and Diverse Ensemblescited by 4
  25. 2025Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shiftcited by 4
  26. 2025CHARM3R: Towards Unseen Camera Height Robust Monocular 3D Detectorcited by 4
  27. 2023Iteratively Learn Diverse Strategies with State Distance Informationcited by 3
  28. 2024AutoFT: Learning an Objective for Robust Fine-Tuningcited by 3
  29. 2024Sharpness-diversity tradeoff: improving flat ensembles with SharpBalancecited by 3
  30. 2022Using Focal Loss to Fight Shallow Heuristics: An Empirical Analysis of Modulated Cross-Entropy in Natural Language Inferencecited by 2
  31. 2023Can Active Sampling Reduce Causal Confusion in Offline Reinforcement Learning?cited by 2
  32. 2024Views Can Be Deceiving: Improved SSL Through Feature Space Augmentationcited by 2
  33. 2024UnLearning from Experience to Avoid Spurious Correlationscited by 2
  34. 2024OOD-Chameleon: Is Algorithm Selection for OOD Generalization Learnable?cited by 2
  35. 2022Using Interventions to Improve Out-of-Distribution Generalization of Text-Matching Recommendation Systemscited by 1
  36. 2023CoinRun: Solving Goal Misgeneralisationcited by 1
  37. 2025Class-Conditional Distribution Balancing for Group Robust Classificationcited by 1
  38. 2025Diversify and Conquer: Open-set Disagreement for Robust Semi-supervised Learning with Outlierscited by 1

50 most cited of 55 All 55 (JSON)