# Probable Domain Generalization via Quantile Risk Minimization

arXiv: https://arxiv.org/abs/2207.09944 · HTML: https://mlsanity.com/paper/2207.09944/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Builds on 85 of its 163 references that are in the map.
- First posted on arXiv 20 Jul 2022.

## Builds on (most cited first, 30 of 85)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [MoleculeNet: A Benchmark for Molecular Machine Learning](https://mlsanity.com/paper/1703.00564/) (2017)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Recognition in Terra Incognita](https://mlsanity.com/paper/1807.04975/) (2018)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Self-Challenging Improves Cross-Domain Generalization](https://mlsanity.com/paper/2007.02454/) (2020)

## Built on by (most cited first, 30 of 66)

- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Nonparametric Identifiability of Causal Representations from Unknown Interventions](https://mlsanity.com/paper/2306.00542/) (2023)
- [ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasets](https://mlsanity.com/paper/2209.00613/) (2022)
- [Domain Generalization in Computational Pathology: Survey and Guidelines](https://mlsanity.com/paper/2310.19656/) (2023)
- [Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data](https://mlsanity.com/paper/2411.10634/) (2024)
- [Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features](https://mlsanity.com/paper/2307.09933/) (2023)
- [DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization](https://mlsanity.com/paper/2308.10285/) (2023)
- [Domain Generalisation via Risk Distribution Matching](https://mlsanity.com/paper/2310.18598/) (2023)
- [Domain Generalisation via Imprecise Learning](https://mlsanity.com/paper/2404.04669/) (2024)
- [Adversarial Training Should Be Cast as a Non-Zero-Sum Game](https://mlsanity.com/paper/2306.11035/) (2023)
- [HYPO: Hyperspherical Out-of-Distribution Generalization](https://mlsanity.com/paper/2402.07785/) (2024)
- [CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysis](https://mlsanity.com/paper/2310.04414/) (2023)
- [Exploring Training on Heterogeneous Data with Mixture of Low-rank Adapters](https://mlsanity.com/paper/2406.09679/) (2024)
- [Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environment](https://mlsanity.com/paper/2406.13371/) (2024)
- [Domain Generalization with Correlated Style Uncertainty](https://mlsanity.com/paper/2212.09950/) (2022)
- [Towards Modality Generalization: A Benchmark and Prospective Analysis](https://mlsanity.com/paper/2412.18277/) (2024)
- [On the Value of Cross-Modal Misalignment in Multimodal Representation Learning](https://mlsanity.com/paper/2504.10143/) (2025)
- [Cross Contrasting Feature Perturbation for Domain Generalization](https://mlsanity.com/paper/2307.12502/) (2023)
- [HCVP: Leveraging Hierarchical Contrastive Visual Prompt for Domain Generalization](https://mlsanity.com/paper/2401.09716/) (2024)
- [AHA: Human-Assisted Out-of-Distribution Generalization and Detection](https://mlsanity.com/paper/2410.08000/) (2024)
- [Wild-Tab: A Benchmark For Out-Of-Distribution Generalization In Tabular Regression](https://mlsanity.com/paper/2312.01792/) (2023)
- [Out-of-Distribution Learning with Human Feedback](https://mlsanity.com/paper/2408.07772/) (2024)
- [Beyond Surface Artifacts: Capturing Shared Latent Forgery Knowledge Across Modalities](https://mlsanity.com/paper/2604.07763/) (2026)
- [The Causal Structure of Domain Invariant Supervised Representation Learning](https://mlsanity.com/paper/2208.06987/) (2022)
- [On the Interventional Kullback-Leibler Divergence](https://mlsanity.com/paper/2302.05380/) (2023)
- [Invariance & Causal Representation Learning: Prospects and Limitations](https://mlsanity.com/paper/2312.03580/) (2023)
- [Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization](https://mlsanity.com/paper/2308.14418/) (2023)
- [INSURE: An Information Theory Inspired Disentanglement and Purification Model for Domain Generalization](https://mlsanity.com/paper/2309.04063/) (2023)
- [Optimal Aggregation of Prediction Intervals under Unsupervised Domain Shift](https://mlsanity.com/paper/2405.10302/) (2024)
- [START: A Generalized State Space Model with Saliency-Driven Token-Aware Transformation](https://mlsanity.com/paper/2410.16020/) (2024)

## Abstract (arXiv)

Domain generalization (DG) seeks predictors which perform well on unseen test distributions by leveraging data drawn from multiple related training distributions or domains. To achieve this, DG is commonly formulated as an average- or worst-case problem over the set of possible domains. However, predictors that perform well on average lack robustness while predictors that perform well in the worst case tend to be overly-conservative. To address this, we propose a new probabilistic framework for DG where the goal is to learn predictors that perform well with high probability. Our key idea is that distribution shifts seen during training should inform us of probable shifts at test time, which we realize by explicitly relating training and test domains as draws from the same underlying meta-distribution. To achieve probable DG, we propose a new optimization problem called Quantile Risk Minimization (QRM). By minimizing the α-quantile of predictor's risk distribution over domains, QRM seeks predictors that perform well with probability α. To solve QRM in practice, we propose the Empirical QRM (EQRM) algorithm and provide: (i) a generalization bound for EQRM; and (ii) the conditions under which EQRM recovers the causal predictor as α → 1. In our experiments, we introduce a more holistic quantile-focused evaluation protocol for DG and demonstrate that EQRM outperforms state-of-the-art baselines on datasets from WILDS and DomainBed.