mlsanityData to 14 Oct 2026

Heterogeneous Risk Minimization

arXiv 2105.03818 ↗First posted 9 May 2021cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

Machine learning algorithms with empirical risk minimization usually suffer from poor generalization performance due to the greedy exploitation of correlations among the training data, which are not stable under distributional shifts. Recently, some invariant learning methods for out-of-distribution (OOD) generalization have been proposed by leveraging multiple training environments to find invariant relationships. However, modern datasets are frequently assembled by merging data from multiple sources without explicit source labels. The resultant unobserved heterogeneity renders many invariant learning methods inapplicable. In this paper, we propose Heterogeneous Risk Minimization (HRM) framework to achieve joint learning of latent heterogeneity among the data and invariant relationship, which leads to stable prediction despite distributional shifts. We theoretically characterize the roles of the environment labels in invariant learning and justify our newly proposed HRM framework. Extensive experimental results validate the effectiveness of our HRM framework.

Key facts. Cited by 90 papers in the map (data through 14 Oct 2026). Builds on 13 of its 24 references that are in the map. First posted on arXiv 9 May 2021.

Builds on

13 of 24 references.

References · 13

  1. 2019Invariant Risk Minimizationcited by 1,854
  2. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,247
  3. 2017Certifying Some Distributional Robustness with Principled Adversarial Trainingcited by 337
  4. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 335
  5. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  6. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310
  7. 2020Environment Inference for Invariant Learningcited by 306
  8. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 291
  9. 2019Task2Vec: Task Embedding for Meta-Learningcited by 205
  10. 2020Invariant Rationalizationcited by 131
  11. 2020Stable Prediction with Model Misspecification and Agnostic Distribution Shiftcited by 62
  12. 2017Causally Regularized Learning with Agnostic Data Selection Biascited by 29
1 more
  1. 2019Incorporating Unlabeled Data into Distributionally Robust Learningcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 23; 2023: 23; 2024: 21; 2025: 13; 2026: 3.

90 papers.

Sort:

By area:

  1. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 436
  2. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  3. 2022Handling Distribution Shifts on Graphs: An Invariance Perspectivecited by 164
  4. 2022Causality Inspired Representation Learning for Domain Generalizationcited by 138
  5. 2021Causal Attention for Unbiased Visual Recognitioncited by 80
  6. 2022ZIN: When and How to Learn Invariance Without Environment Partition?cited by 58
  7. 2022Out-of-distribution Generalization with Causal Invariant Transformationscited by 45
  8. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 44
  9. 2023Does Invariant Graph Learning via Environment Augmentation Learn Invariance?cited by 42
  10. 2022Unleashing the Power of Graph Data Augmentation on Covariate Distribution Shiftcited by 38
  11. 2022Identifying Weight-Variant Latent Causal Modelscited by 34
  12. 2023Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalizationcited by 29
38 more
  1. 2024Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learningcited by 27
  2. 2023Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domainscited by 24
  3. 2023Invariant Learning via Probability of Sufficient and Necessary Causescited by 22
  4. 2021Kernelized Heterogeneous Risk Minimizationcited by 21
  5. 2024A Survey on Evaluation of Out-of-Distribution Generalizationcited by 21
  6. 2023Rethinking the Evaluation Protocol of Domain Generalizationcited by 20
  7. 2021A Theoretical Analysis on Independence-driven Importance Weighting for Covariate-shift Generalizationcited by 19
  8. 2022When Does Group Invariant Learning Survive Spurious Correlations?cited by 18
  9. 2022Exploring Optimal Substructure for Out-of-distribution Generalization via Feature-targeted Model Pruningcited by 17
  10. 2022Learning to Split for Automatic Bias Detectioncited by 16
  11. 2023Universal Domain Adaptation via Compressive Attention Matchingcited by 15
  12. 2024Domain Generalisation via Imprecise Learningcited by 14
  13. 2022Stable Learning via Sparse Variable Independencecited by 13
  14. 2024Multimodality Invariant Learning for Multimedia-Based New Item Recommendationcited by 13
  15. 2022An Empirical Study on Distribution Shift Robustness From the Perspective of Pre-Training and Data Augmentationcited by 11
  16. 2022Towards out of distribution generalization for problems in mechanicscited by 11
  17. 2024Towards Robust Out-of-Distribution Generalization Bounds via Sharpnesscited by 11
  18. 2023Graph Structure and Feature Extrapolation for Out-of-Distribution Generalizationcited by 10
  19. 2024Understanding Domain Generalization: A Noise Robustness Perspectivecited by 8
  20. 2024Distributionally Generative Augmentation for Fair Facial Attribute Classificationcited by 8
  21. 2022Breaking Correlation Shift via Conditional Invariant Regularizercited by 7
  22. 2023Generalized Universal Domain Adaptation with Generative Flow Networkscited by 7
  23. 2023Topology-aware Robust Optimization for Out-of-distribution Generalizationcited by 7
  24. 2023Mitigating Simplicity Bias in Deep Learning for Improved OOD Generalization and Robustnesscited by 7
  25. 2021Learning Stable Classifiers by Transferring Unstable Featurescited by 6
  26. 2022Class Is Invariant to Context and Vice Versa: On Learning Invariance for Out-Of-Distribution Generalizationcited by 6
  27. 2024Improving out-of-distribution generalization in graphs via hierarchical semantic environmentscited by 5
  28. 2024Invariant Risk Minimization Is A Total Variation Modelcited by 5
  29. 2024Scalable Out-of-distribution Robustness in the Presence of Unobserved Confounderscited by 5
  30. 2022Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learningcited by 4
  31. 2023Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalizationcited by 4
  32. 2024STAT: Towards Generalizable Temporal Action Localizationcited by 4
  33. 2025TIF: Learning Temporal Invariance in Android Malware Detectorscited by 4
  34. 2022Enhancing Distributional Stability among Sub-populationscited by 3
  35. 2022Decorr: Environment Partitioning for Invariant Learning and OOD Generalizationcited by 3
  36. 2023A Causal Framework to Unify Common Domain Generalization Approachescited by 3
  37. 2023Pareto Invariant Representation Learning for Multimedia Recommendationcited by 3
  38. 2023Environment Diversification with Multi-head Neural Network for Invariant Learningcited by 3

50 most cited of 90 All 90 (JSON)