mlsanityData to 7 Oct 2026

Kernelized Heterogeneous Risk Minimization

arXiv 2110.12425 ↗First posted 24 Oct 2021cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

The ability to generalize under distributional shifts is essential to reliable machine learning, while models optimized with empirical risk minimization usually fail on non-i.i.d testing data. Recently, invariant learning methods for out-of-distribution (OOD) generalization propose to find causally invariant relationships with multi-environments. However, modern datasets are frequently multi-sourced without explicit source labels, rendering many invariant learning methods inapplicable. In this paper, we propose Kernelized Heterogeneous Risk Minimization (KerHRM) algorithm, which achieves both the latent heterogeneity exploration and invariant learning in kernel space, and then gives feedback to the original neural network by appointing invariant gradient direction. We theoretically justify our algorithm and empirically validate the effectiveness of our algorithm with extensive experiments.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 24 Oct 2021.

Builds on

15 of 24 references.

References · 15

  1. 2019Invariant Risk Minimizationcited by 1,832
  2. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  3. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  4. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  5. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 333
  6. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  7. 2020Environment Inference for Invariant Learningcited by 306
  8. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  9. 2020The Risks of Invariant Risk Minimizationcited by 262
  10. 2020Invariant Rationalizationcited by 131
  11. 2021Heterogeneous Risk Minimizationcited by 90
  12. 2019Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobiancited by 67
3 more
  1. 2020Stable Prediction with Model Misspecification and Agnostic Distribution Shiftcited by 62
  2. 2016Does Distributionally Robust Supervised Learning Give Robust Classifiers?cited by 38
  3. 2019Incorporating Unlabeled Data into Distributionally Robust Learningcited by 20

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 7; 2023: 7; 2024: 4; 2025: 2.

21 papers.

Sort:

  1. 2022ZIN: When and How to Learn Invariance Without Environment Partition?cited by 58
  2. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 20
  3. 2021A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalizationcited by 18
  4. 2023Universal Domain Adaptation via Compressive Attention Matchingcited by 15
  5. 2022Stable Learning via Sparse Variable Independencecited by 13
  6. 2022An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentationcited by 11
  7. 2022Towards out of distribution generalization for problems in mechanicscited by 11
  8. 2024Distributionally Generative Augmentation for Fair Facial Attribute Classificationcited by 8
  9. 2023Generalized Universal Domain Adaptation with Generative Flow Networkscited by 7
  10. 2024Improving out-of-distribution generalization in graphs via hierarchical semantic environmentscited by 5
  11. 2022Supervision Adaptation Balancing In-distribution Generalization and Out-of-distribution Detectioncited by 4
  12. 2023Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalizationcited by 4
9 more
  1. 2023Pareto Invariant Representation Learning for Multimedia Recommendationcited by 3
  2. 2023Environment Diversification with Multi-head Neural Network for Invariant Learningcited by 3
  3. 2022Invariance Learning based on Label Hierarchycited by 1
  4. 2023Exploring and Exploiting Data Heterogeneity in Recommendationcited by 1
  5. 2023Out-of-Distribution Optimality of Invariant Risk Minimizationcited by 1
  6. 2025Class-Conditional Distribution Balancing for Group Robust Classificationcited by 1
  7. 2022A Survey on Causal Representation Learning and Future Work for Medical Image Analysiscited by 0
  8. 2024Invariant Correlation of Representation with Label: Enhancing Domain Generalization in Noisy Environmentscited by 0
  9. 2025Data Heterogeneity Modeling for Trustworthy Machine Learningcited by 0