# Risk Variance Penalization

arXiv: https://arxiv.org/abs/2006.07544 · HTML: https://mlsanity.com/paper/2006.07544/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2020.

## Built on by (most cited first, 21 of 21)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Discovering Invariant Rationales for Graph Neural Networks](https://mlsanity.com/paper/2201.12872/) (2022)
- [Handling Distribution Shifts on Graphs: An Invariance Perspective](https://mlsanity.com/paper/2202.02466/) (2022)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?](https://mlsanity.com/paper/2106.02890/) (2021)
- [ZIN: When and How to Learn Invariance Without Environment Partition?](https://mlsanity.com/paper/2203.05818/) (2022)
- [Model Agnostic Sample Reweighting for Out-of-Distribution Learning](https://mlsanity.com/paper/2301.09819/) (2023)
- [An Empirical Framework for Domain Generalization in Clinical Settings](https://mlsanity.com/paper/2103.11163/) (2021)
- [Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments](https://mlsanity.com/paper/2106.09913/) (2021)
- [NAS-OoD: Neural Architecture Search for Out-of-Distribution Generalization](https://mlsanity.com/paper/2109.02038/) (2021)
- [An Information-theoretic Approach to Distribution Shifts](https://mlsanity.com/paper/2106.03783/) (2021)
- [Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization](https://mlsanity.com/paper/2303.00442/) (2023)
- [Optimization-based Causal Estimation from Heterogenous Environments](https://mlsanity.com/paper/2109.11990/) (2021)
- [Core Risk Minimization using Salient ImageNet](https://mlsanity.com/paper/2203.15566/) (2022)
- [Are Bias Mitigation Techniques for Deep Learning Effective?](https://mlsanity.com/paper/2104.00170/) (2021)
- [Out-of-Distribution Generalization Analysis via Influence Function](https://mlsanity.com/paper/2101.08521/) (2021)
- [Continuous Invariance Learning](https://mlsanity.com/paper/2310.05348/) (2023)
- [Learning Under Adversarial and Interventional Shifts](https://mlsanity.com/paper/2103.15933/) (2021)
- [Antibody DomainBed: Out-of-Distribution Generalization in Therapeutic Protein Design](https://mlsanity.com/paper/2407.21028/) (2024)
- [Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule](https://mlsanity.com/paper/2408.14126/) (2024)
- [Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.07798/) (2022)

## Abstract (arXiv)

The key of the out-of-distribution (OOD) generalization is to generalize invariance from training domains to target domains. The variance risk extrapolation (V-REx) is a practical OOD method, which depends on a domain-level regularization but lacks theoretical verifications about its motivation and utility. This article provides theoretical insights into V-REx by studying a variance-based regularizer. We propose Risk Variance Penalization (RVP), which slightly changes the regularization of V-REx but addresses the theory concerns about V-REx. We provide theoretical explanations and a theory-inspired tuning scheme for the regularization parameter of RVP. Our results point out that RVP discovers a robust predictor. Finally, we experimentally show that the proposed regularizer can find an invariant predictor under certain conditions.