mlsanityData to 14 Oct 2026

Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detection

arXiv 2306.09158 ↗First posted 15 Jun 2023cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

Modern machine learning models deployed in the wild can encounter both covariate and semantic shifts, giving rise to the problems of out-of-distribution (OOD) generalization and OOD detection respectively. While both problems have received significant research attention lately, they have been pursued independently. This may not be surprising, since the two tasks have seemingly conflicting goals. This paper provides a new unified approach that is capable of simultaneously generalizing to covariate shifts while robustly detecting semantic shifts. We propose a margin-based learning framework that exploits freely available unlabeled data in the wild that captures the environmental test-time OOD distributions under both covariate and semantic shifts. We show both empirically and theoretically that the proposed margin constraint is the key to achieving both OOD generalization and detection. Extensive experiments show the superiority of our framework, outperforming competitive baselines that specialize in either OOD generalization or OOD detection. Code is publicly available at https://github.com/deeplearning-wisc/scone.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 60 of its 99 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

60 of 99 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2016Wide Residual Networkscited by 3,070
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  7. 2013Describing Textures in the Wildcited by 2,125
  8. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  9. 2019Invariant Risk Minimizationcited by 1,854
  10. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  11. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  12. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
38 more
  1. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  2. 2017Adversarial Discriminative Domain Adaptationcited by 1,311
  3. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  4. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,200
  5. 2020In Search of Lost Domain Generalizationcited by 1,054
  6. 2017Deeper, Broader and Artier Domain Generalizationcited by 986
  7. 2018Deep Anomaly Detection with Outlier Exposurecited by 942
  8. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
  9. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  10. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  11. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  12. 2013Domain Generalization via Invariant Feature Representationcited by 623
  13. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  14. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  15. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  16. 2020Self-Challenging Improves Cross-Domain Generalizationcited by 426
  17. 2019Contrastive Adaptation Network for Unsupervised Domain Adaptationcited by 407
  18. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 405
  19. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 339
  20. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  21. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  22. 2019Adversarial Domain Adaptation with Domain Mixupcited by 250
  23. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  24. 2019Learning De-biased Representations with Biased Representationscited by 199
  25. 2020Invariant Risk Minimization Gamescited by 188
  26. 2020Adaptive Risk Minimization: Learning to Adapt to Domain Shiftcited by 183
  27. 2017Domain Generalization by Marginal Transfer Learningcited by 180
  28. 2020Universal Domain Adaptation through Self Supervisioncited by 172
  29. 2020Understanding Self-Training for Gradual Domain Adaptationcited by 149
  30. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 138
  31. 2022POEM: Out-of-Distribution Detection with Posterior Samplingcited by 108
  32. 2020Heterogeneous Domain Generalization via Domain Mixupcited by 103
  33. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  34. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  35. 2022Training OOD Detectors in their Natural Habitatscited by 86
  36. 2021What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theorycited by 60
  37. 2022Unified Optimal Transport Framework for Universal Domain Adaptationcited by 57
  38. 2020In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustnesscited by 50

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 31; 2025: 16; 2026: 5.

57 papers.

Sort:

By area:

  1. 2024ARGS: Alignment as Reward-Guided Searchcited by 103
  2. 2024HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detectioncited by 99
  3. 2024Process Reward Model with Q-Value Rankingscited by 80
  4. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  5. 2024How Does Unlabeled Data Provably Help Out-of-Distribution Detection?cited by 36
  6. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  7. 2024COME: Test-time adaption by Conservatively Minimizing Entropycited by 31
  8. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 26
  9. 2023A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learningcited by 22
  10. 2024Unified Entropy Optimization for Open-Set Test-Time Adaptationcited by 19
  11. 2025OODD: Test-time Out-of-Distribution Detection with Dynamic Dictionarycited by 17
  12. 2023How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?cited by 15
38 more
  1. 2024FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detectioncited by 14
  2. 2025Handling Out-of-Distribution Data: A Surveycited by 14
  3. 2024Pursuing Feature Separation based on Neural Collapse for Out-of-Distribution Detectioncited by 12
  4. 2025Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learningcited by 11
  5. 2024SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximationcited by 10
  6. 2024On the Learnability of Out-of-distribution Detectioncited by 8
  7. 2024Adaptive Hierarchical Graph Cut for Multi-granularity Out-of-distribution Detectioncited by 8
  8. 2024AHA: Human-Assisted Out-of-Distribution Generalization and Detectioncited by 7
  9. 2024CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detectioncited by 6
  10. 2024When and How Does In-Distribution Label Help Out-of-Distribution Detection?cited by 6
  11. 2024Out-of-Distribution Learning with Human Feedbackcited by 6
  12. 2024GDDA: Semantic OOD Detection on Graphs under Covariate Shift via Score-Based Diffusion Modelscited by 5
  13. 2024MADOD: Generalizing OOD Detection to Unseen Domains via G-Invariance Meta-Learningcited by 5
  14. 2024Deep Active Learning in the Open Worldcited by 5
  15. 2024QUTE: Quantifying Uncertainty in TinyML with Early-exit-assisted ensembles for model-monitoringcited by 4
  16. 2024What If the Input is Expanded in OOD Detection?cited by 4
  17. 2025Out-of-Distribution Detection using Synthetic Data Generationcited by 4
  18. 2024Bridging OOD Detection and Generalization: A Graph-Theoretic Viewcited by 3
  19. 2025Open-Insect: Benchmarking Open-Set Recognition of Novel Species in Biodiversity Monitoringcited by 3
  20. 2025Enhancing Out-of-Distribution Detection with Extended Logit Normalizationcited by 3
  21. 2025Temp-SCONE: A Novel Out-of-Distribution Detection and Domain Generalization Framework for Wild Data with Temporal Shiftcited by 3
  22. 2026T-QPM: Enabling Temporal Out-Of-Distribution Detection and Domain Generalization for Vision-Language Models in Open-Worldcited by 3
  23. 2023Why Train More? Effective and Efficient Membership Inference via Memorizationcited by 2
  24. 2024Monitizer: Automating Design and Evaluation of Neural Network Monitorscited by 2
  25. 2024Edge Case Detection in Automated Driving: Methods, Challenges, and Future Directionscited by 2
  26. 2024FEVER-OOD: Free Energy Vulnerability Elimination for Robust Out-of-Distribution Detectioncited by 2
  27. 2025FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Modelscited by 2
  28. 2025Medix: Out-of-Distribution Detection from Unlabeled Wild Data via Robust Gradient Statisticscited by 2
  29. 2023Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theorycited by 1
  30. 2024Introducing 'Inside' Out of Distributioncited by 1
  31. 2024Feature-Space Semantic Invariance: Enhanced OOD Detection for Open-Set Domain Generalizationcited by 1
  32. 2025Where's the liability in the Generative Era? Recovery-based Black-Box Detection of AI-Generated Contentcited by 1
  33. 2025TerraIncognita: A Dynamic Benchmark for Species Discovery Using Frontier Modelscited by 1
  34. 2025Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domainscited by 1
  35. 2025Delta Energy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalizationcited by 1
  36. 2026One Model, Many Behaviors: Training-Induced Effects on Out-of-Distribution Detectioncited by 1
  37. 2024Enhancing Anomaly Detection Generalization through Knowledge Exposure: The Dual Effects of Augmentationcited by 0
  38. 2024Hallucination Detection in Virtually-Stained Histology: A Latent Space Baselinecited by 0

50 most cited of 57 All 57 (JSON)