# Kernelized Heterogeneous Risk Minimization

arXiv: https://arxiv.org/abs/2110.12425 · HTML: https://mlsanity.com/paper/2110.12425/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 24 references that are in the map.
- First posted on arXiv 24 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [Learning Models with Uniform Performance via Distributionally Robust Optimization](https://mlsanity.com/paper/1810.08750/) (2018)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [Environment Inference for Invariant Learning](https://mlsanity.com/paper/2010.07249/) (2020)
- [Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations](https://mlsanity.com/paper/1505.05116/) (2015)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Invariant Rationalization](https://mlsanity.com/paper/2003.09772/) (2020)
- [Heterogeneous Risk Minimization](https://mlsanity.com/paper/2105.03818/) (2021)
- [Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian](https://mlsanity.com/paper/1906.05392/) (2019)
- [Stable Prediction with Model Misspecification and Agnostic Distribution Shift](https://mlsanity.com/paper/2001.11713/) (2020)
- [Does Distributionally Robust Supervised Learning Give Robust Classifiers?](https://mlsanity.com/paper/1611.02041/) (2016)
- [Incorporating Unlabeled Data into Distributionally Robust Learning](https://mlsanity.com/paper/1912.07729/) (2019)

## Built on by (most cited first, 21 of 21)

- [ZIN: When and How to Learn Invariance Without Environment Partition?](https://mlsanity.com/paper/2203.05818/) (2022)
- [A Survey on Evaluation of Out-of-Distribution Generalization](https://mlsanity.com/paper/2403.01874/) (2024)
- [A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalization](https://mlsanity.com/paper/2111.02355/) (2021)
- [Universal Domain Adaptation via Compressive Attention Matching](https://mlsanity.com/paper/2304.11862/) (2023)
- [Stable Learning via Sparse Variable Independence](https://mlsanity.com/paper/2212.00992/) (2022)
- [An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentation](https://mlsanity.com/paper/2205.12753/) (2022)
- [Towards out of distribution generalization for problems in mechanics](https://mlsanity.com/paper/2206.14917/) (2022)
- [Distributionally Generative Augmentation for Fair Facial Attribute Classification](https://mlsanity.com/paper/2403.06606/) (2024)
- [Generalized Universal Domain Adaptation with Generative Flow Networks](https://mlsanity.com/paper/2305.04466/) (2023)
- [Improving out-of-distribution generalization in graphs via hierarchical semantic environments](https://mlsanity.com/paper/2403.01773/) (2024)
- [Supervision Adaptation Balancing In-distribution Generalization and Out-of-distribution Detection](https://mlsanity.com/paper/2206.09380/) (2022)
- [Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalization](https://mlsanity.com/paper/2305.15889/) (2023)
- [Pareto Invariant Representation Learning for Multimedia Recommendation](https://mlsanity.com/paper/2308.04706/) (2023)
- [Environment Diversification with Multi-head Neural Network for Invariant Learning](https://mlsanity.com/paper/2308.08778/) (2023)
- [Invariance Learning based on Label Hierarchy](https://mlsanity.com/paper/2203.15549/) (2022)
- [Exploring and Exploiting Data Heterogeneity in Recommendation](https://mlsanity.com/paper/2305.15431/) (2023)
- [Out-of-Distribution Optimality of Invariant Risk Minimization](https://mlsanity.com/paper/2307.11972/) (2023)
- [Class-Conditional Distribution Balancing for Group Robust Classification](https://mlsanity.com/paper/2504.17314/) (2025)
- [A Survey on Causal Representation Learning and Future Work for Medical Image Analysis](https://mlsanity.com/paper/2210.16034/) (2022)
- [Invariant Correlation of Representation with Label: Enhancing Domain Generalization in Noisy Environments](https://mlsanity.com/paper/2407.01749/) (2024)
- [Data Heterogeneity Modeling for Trustworthy Machine Learning](https://mlsanity.com/paper/2506.00969/) (2025)

## Abstract (arXiv)

The ability to generalize under distributional shifts is essential to reliable machine learning, while models optimized with empirical risk minimization usually fail on non-i.i.d testing data. Recently, invariant learning methods for out-of-distribution (OOD) generalization propose to find causally invariant relationships with multi-environments. However, modern datasets are frequently multi-sourced without explicit source labels, rendering many invariant learning methods inapplicable. In this paper, we propose Kernelized Heterogeneous Risk Minimization (KerHRM) algorithm, which achieves both the latent heterogeneity exploration and invariant learning in kernel space, and then gives feedback to the original neural network by appointing invariant gradient direction. We theoretically justify our algorithm and empirically validate the effectiveness of our algorithm with extensive experiments.