# Enforcing Predictive Invariance across Structured Biomedical Domains

arXiv: https://arxiv.org/abs/2006.03908 · HTML: https://mlsanity.com/paper/2006.03908/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2020.

## Built on by (most cited first, 23 of 23)

- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.05265/) (2021)
- [Towards a Theoretical Framework of Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.04496/) (2021)
- [Invariant Information Bottleneck for Domain Generalization](https://mlsanity.com/paper/2106.06333/) (2021)
- [Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?](https://mlsanity.com/paper/2106.02890/) (2021)
- [ZIN: When and How to Learn Invariance Without Environment Partition?](https://mlsanity.com/paper/2203.05818/) (2022)
- [Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments](https://mlsanity.com/paper/2106.09913/) (2021)
- [Out-of-distribution Prediction with Invariant Risk Minimization: The Limitation and An Effective Fix](https://mlsanity.com/paper/2101.07732/) (2021)
- [Invariant Learning via Probability of Sufficient and Necessary Causes](https://mlsanity.com/paper/2309.12559/) (2023)
- [Invariance Makes LLM Unlearning Resilient Even to Unanticipated Downstream Fine-Tuning](https://mlsanity.com/paper/2506.01339/) (2025)
- [Learning Representations that Support Robust Transfer of Predictors](https://mlsanity.com/paper/2110.09940/) (2021)
- [Meta-Causal Feature Learning for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.10156/) (2022)
- [Learning for Counterfactual Fairness from Observational Data](https://mlsanity.com/paper/2307.08232/) (2023)
- [ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representations](https://mlsanity.com/paper/2303.01092/) (2023)
- [What Is Missing in IRM Training and Evaluation? Challenges and Solutions](https://mlsanity.com/paper/2303.02343/) (2023)
- [Causal Learning for Socially Responsible AI](https://mlsanity.com/paper/2104.12278/) (2021)
- [Continuous Invariance Learning](https://mlsanity.com/paper/2310.05348/) (2023)
- [Learning Optimal Features via Partial Invariance](https://mlsanity.com/paper/2301.12067/) (2023)
- [Domain Generalization for Vision-based Driving Trajectory Generation](https://mlsanity.com/paper/2109.13858/) (2021)
- [Pruning Spurious Subgraphs for Graph Out-of-Distribution Generalization](https://mlsanity.com/paper/2506.05957/) (2025)
- [Bridging Distribution Shift and AI Safety: Conceptual and Methodological Synergies](https://mlsanity.com/paper/2505.22829/) (2025)
- [Revisiting Spurious Correlation in Domain Generalization](https://mlsanity.com/paper/2406.11517/) (2024)
- [Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2208.07798/) (2022)

## Abstract (arXiv)

Many biochemical applications such as molecular property prediction require models to generalize beyond their training domains (environments). Moreover, natural environments in these tasks are structured, defined by complex descriptors such as molecular scaffolds or protein families. Therefore, most environments are either never seen during training, or contain only a single training example. To address these challenges, we propose a new regret minimization (RGM) algorithm and its extension for structured environments. RGM builds from invariant risk minimization (IRM) by recasting simultaneous optimality condition in terms of predictive regret, finding a representation that enables the predictor to compete against an oracle with hindsight access to held-out environments. The structured extension adaptively highlights variation due to complex environments via specialized domain perturbations. We evaluate our method on multiple applications: molecular property prediction, protein homology and stability prediction and show that RGM significantly outperforms previous state-of-the-art baselines.