# Invariant Learning via Probability of Sufficient and Necessary Causes

arXiv: https://arxiv.org/abs/2309.12559 · HTML: https://mlsanity.com/paper/2309.12559/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 79 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 30 of 43)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Towards Out-Of-Distribution Generalization: A Survey](https://mlsanity.com/paper/2108.13624/) (2021)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [What Should Not Be Contrastive in Contrastive Learning](https://mlsanity.com/paper/2008.05659/) (2020)
- [Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2106.06607/) (2021)
- [Invariant Risk Minimization Games](https://mlsanity.com/paper/2002.04692/) (2020)
- [Improve Unsupervised Domain Adaptation with Mixup Training](https://mlsanity.com/paper/2001.00677/) (2020)
- [Domain Generalization via Conditional Invariant Representation](https://mlsanity.com/paper/1807.08479/) (2018)
- [Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphs](https://mlsanity.com/paper/2202.05441/) (2022)
- [Invariant Causal Prediction for Nonlinear Models](https://mlsanity.com/paper/1706.08576/) (2017)
- [Domain Adaptation by Using Causal Inference to Predict Invariant Conditional Distributions](https://mlsanity.com/paper/1707.06422/) (2017)
- [Anchor regression: heterogeneous data meets causality](https://mlsanity.com/paper/1801.06229/) (2018)
- [Learning Independent Causal Mechanisms](https://mlsanity.com/paper/1712.00961/) (2017)
- [Heterogeneous Domain Generalization via Domain Mixup](https://mlsanity.com/paper/2009.05448/) (2020)
- [Understanding Contrastive Learning Requires Incorporating Inductive Biases](https://mlsanity.com/paper/2202.14037/) (2022)
- [Heterogeneous Risk Minimization](https://mlsanity.com/paper/2105.03818/) (2021)

## Built on by (most cited first, 22 of 22)

- [Causal Evaluation of Language Models](https://mlsanity.com/paper/2405.00622/) (2024)
- [Feature Attribution with Necessity and Sufficiency via Dual-stage Perturbation Test for Causal Explanation](https://mlsanity.com/paper/2402.08845/) (2024)
- [Towards the Causal Complete Cause of Multi-Modal Representation Learning](https://mlsanity.com/paper/2407.14058/) (2024)
- [When Graph Neural Network Meets Causality: Opportunities, Methodologies and An Outlook](https://mlsanity.com/paper/2312.12477/) (2023)
- [TIF: Learning Temporal Invariance in Android Malware Detectors](https://mlsanity.com/paper/2502.05098/) (2025)
- [Curious Causality-Seeking Agents Learn Meta Causal World](https://mlsanity.com/paper/2506.23068/) (2025)
- [Unifying Invariance and Spuriousity for Graph Out-of-Distribution via Probability of Necessity and Sufficiency](https://mlsanity.com/paper/2402.09165/) (2024)
- [Learning Invariant Causal Mechanism from Vision-Language Models](https://mlsanity.com/paper/2405.15289/) (2024)
- [Out-of-distribution Generalization for Total Variation based Invariant Risk Minimization](https://mlsanity.com/paper/2502.19665/) (2025)
- [Unifying Invariant and Variant Features for Graph Out-of-Distribution via Probability of Necessity and Sufficiency](https://mlsanity.com/paper/2407.15273/) (2024)
- [Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2408.01697/) (2024)
- [Medical Image Quality Assessment based on Probability of Necessity and Sufficiency](https://mlsanity.com/paper/2410.08118/) (2024)
- [Revisiting Spurious Correlation in Domain Generalization](https://mlsanity.com/paper/2406.11517/) (2024)
- [Invariant Causal Routing for Governing Social Norms in Online Market Economies](https://mlsanity.com/paper/2603.04534/) (2026)
- [Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment](https://mlsanity.com/paper/2404.19567/) (2024)
- [Seeking the Sufficiency and Necessity Causal Features in Multimodal Representation Learning](https://mlsanity.com/paper/2408.16577/) (2024)
- [Enhancing Few-Shot Out-of-Distribution Detection with Gradient Aligned Context Optimization](https://mlsanity.com/paper/2411.15736/) (2024)
- [Integrating Causal Inference with Graph Neural Networks for Alzheimer's Disease Analysis](https://mlsanity.com/paper/2511.14922/) (2025)
- [Causal-Driven Feature Evaluation for Cross-Domain Image Classification](https://mlsanity.com/paper/2601.20176/) (2026)
- [Seeking Necessary and Sufficient Information from Multimodal Medical Data](https://mlsanity.com/paper/2603.00289/) (2026)
- [Causally Sufficient and Necessary Feature Expansion for Class-Incremental Learning](https://mlsanity.com/paper/2603.09145/) (2026)
- [Beyond Sufficiency: Time Series Explanation with Counterfactual Necessity](https://mlsanity.com/paper/2607.21573/) (2026)

## Abstract (arXiv)

Out-of-distribution (OOD) generalization is indispensable for learning models in the wild, where testing distribution typically unknown and different from the training. Recent methods derived from causality have shown great potential in achieving OOD generalization. However, existing methods mainly focus on the invariance property of causes, while largely overlooking the property of textitsufficiency and textitnecessity conditions. Namely, a necessary but insufficient cause (feature) is invariant to distribution shift, yet it may not have required accuracy. By contrast, a sufficient yet unnecessary cause (feature) tends to fit specific data well but may have a risk of adapting to a new domain. To capture the information of sufficient and necessary causes, we employ a classical concept, the probability of sufficiency and necessary causes (PNS), which indicates the probability of whether one is the necessary and sufficient cause. To associate PNS with OOD generalization, we propose PNS risk and formulate an algorithm to learn representation with a high PNS value. We theoretically analyze and prove the generalizability of the PNS risk. Experiments on both synthetic and real-world benchmarks demonstrate the effectiveness of the proposed method. The details of the implementation can be found at the GitHub repository: https://github.com/ymy4323460/CaSN.