mlsanityData to 7 Oct 2026

Out of Distribution Generalization in Machine Learning

arXiv 2103.02667 ↗First posted 3 Mar 2021stat.ML cs.LGcited by 59

Not on the map. Why?

Abstract (arXiv)

Machine learning has achieved tremendous success in a variety of domains in recent years. However, a lot of these success stories have been in places where the training and the testing distributions are extremely similar to each other. In everyday situations when models are tested in slightly different data than they were trained on, ML algorithms can fail spectacularly. This research attempts to formally define this problem, what sets of assumptions are reasonable to make in our data and what kind of guarantees we hope to obtain from them. Then, we focus on a certain class of out of distribution problems, their assumptions, and introduce simple algorithms that follow from these assumptions that are able to provide more reliable generalization. A central topic in the thesis is the strong link between discovering the causal structure of the data, finding features that are reliable (when using them to predict) regardless of their context, and out of distribution generalization.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2021.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 12; 2023: 10; 2024: 10; 2025: 10; 2026: 3.

59 papers.

Sort:

By area:

  1. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 122
  2. 2021OoD-Bench: Quantifying and Understanding Two Dimensions of Out-of-Distribution Generalizationcited by 86
  3. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learningcited by 71
  4. 2024Dual-Personalizing Adapter for Federated Foundation Modelscited by 54
  5. 2021Attention, please! A survey of Neural Attention Models in Deep Learningcited by 50
  6. 2022Out-of-distribution generalization for learning quantum dynamicscited by 49
  7. 2022Invariant Feature Learning for Generalized Long-Tailed Classificationcited by 34
  8. 2023Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Modelscited by 32
  9. 2021Dangers of Bayesian Model Averaging under Covariate Shiftcited by 31
  10. 2022Gradient Masked Averaging for Federated Learningcited by 22
  11. 2021Out-of-Distribution Generalization in Kernel Regressioncited by 21
  12. 2023Masked Images Are Counterfactual Samples for Robust Fine-tuningcited by 20
38 more
  1. 2021Optimization-based Causal Estimation from Heterogenous Environmentscited by 16
  2. 2022Dimensionless machine learning: Imposing exact units equivariancecited by 16
  3. 2024Identifiable Exchangeable Mechanisms for Causal Structure and Representation Learningcited by 16
  4. 2022Meta-Causal Feature Learning for Out-of-Distribution Generalizationcited by 14
  5. 2024Domain Generalisation via Imprecise Learningcited by 14
  6. 2025When can in-context learning generalize out of task distribution?cited by 13
  7. 2023Subset verification and search algorithms for causal DAGscited by 12
  8. 2021Contrastive Learning of Generalized Game Representationscited by 11
  9. 2021Towards a theory of out-of-distribution learningcited by 11
  10. 2025Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advancescited by 11
  11. 2021On Invariance Penalties for Risk Minimizationcited by 10
  12. 2023Bad Habits: Policy Confounding and Out-of-Trajectory Generalization in RLcited by 10
  13. 2024Neural Context Flows for Meta-Learning of Dynamical Systemscited by 9
  14. 2025When and How Does CLIP Enable Domain and Compositional Generalization?cited by 9
  15. 2021Scaling Laws for the Few-Shot Adaptation of Pre-trained Image Classifierscited by 8
  16. 2023Group-Conditional Conformal Prediction via Quantile Regression Calibration for Crop and Weed Classificationcited by 8
  17. 2022Prospective Learning: Principled Extrapolation to the Futurecited by 6
  18. 2023In Search of Deep Learning Architectures for Load Forecasting: A Comparative Analysis and the Impact of the Covid-19 Pandemic on Model Performancecited by 5
  19. 2024Generalising realisability in statistical learning theory under epistemic uncertaintycited by 5
  20. 2025Weight-Space Linear Recurrent Neural Networkscited by 5
  21. 2021A topological solution to object segmentation and trackingcited by 4
  22. 2022(Im)possibility of Collective Intelligencecited by 4
  23. 2022Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithmscited by 4
  24. 2022Contrastive Retrospection: honing in on critical steps for rapid learning and generalization in RLcited by 4
  25. 2024Prospective Learning: Learning for a Dynamic Futurecited by 4
  26. 2025Distributionally Robust Optimization via Diffusion Ambiguity Modelingcited by 4
  27. 2023Consistency Regularization for Domain Generalization with Logit Attribution Matchingcited by 3
  28. 2023A Causal Framework to Unify Common Domain Generalization Approachescited by 3
  29. 2024Towards Understanding Variants of Invariant Risk Minimization through the Lens of Calibrationcited by 3
  30. 2024Demystifying Spectral Bias on Real-World Datacited by 3
  31. 2026Distributionally Robust Optimization via Generative Ambiguity Modelingcited by 3
  32. 2021SHIFT15M: Fashion-specific dataset for set-to-set matching with several distribution shiftscited by 2
  33. 2022Influence-Augmented Local Simulators: A Scalable Solution for Fast Deep RL in Large Networked Systemscited by 2
  34. 2023Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playcited by 2
  35. 2024NeuralOOD: Improving Out-of-Distribution Generalization Performance with Brain-machine Fusion Learning Frameworkcited by 2
  36. 2025Federated Adapter on Foundation Models: An Out-Of-Distribution Approachcited by 2
  37. 2021Exploration of Dark Chemical Genomics Space via Portal Learning: Applied to Targeting the Undruggable Genome and COVID-19 Anti-Infective Polypharmacologycited by 1
  38. 2022Electricity Consumption Forecasting for Out-of-distribution Time-of-Use Tariffscited by 1

50 most cited of 59 All 59 (JSON)