# CGLearn: Consistent Gradient-Based Learning for Out-of-Distribution Generalization

arXiv: https://arxiv.org/abs/2411.06040 · HTML: https://mlsanity.com/paper/2411.06040/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 9 Nov 2024.

## Builds on (most cited first, 27 of 27)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [DAGs with NO TEARS: Continuous Optimization for Structure Learning](https://mlsanity.com/paper/1803.01422/) (2018)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [DAG-GNN: DAG Structure Learning with Graph Neural Networks](https://mlsanity.com/paper/1904.10098/) (2019)
- [An Investigation of Why Overparameterization Exacerbates Spurious Correlations](https://mlsanity.com/paper/2005.04345/) (2020)
- [The Risks of Invariant Risk Minimization](https://mlsanity.com/paper/2010.05761/) (2020)
- [Gradient Matching for Domain Generalization](https://mlsanity.com/paper/2104.09937/) (2021)
- [Gradient-Based Neural DAG Learning](https://mlsanity.com/paper/1906.02226/) (2019)
- [Learning Sparse Nonparametric DAGs](https://mlsanity.com/paper/1909.13189/) (2019)
- [On the Role of Sparsity and DAG Constraints for Learning Linear DAGs](https://mlsanity.com/paper/2006.10201/) (2020)
- [Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy To Game](https://mlsanity.com/paper/2102.13647/) (2021)
- [Learning explanations that are hard to vary](https://mlsanity.com/paper/2009.00329/) (2020)
- [Masked Gradient-Based Causal Structure Learning](https://mlsanity.com/paper/1910.08527/) (2019)
- [Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Models](https://mlsanity.com/paper/2110.07736/) (2021)
- [On the Impact of Spurious Correlation for Out-of-distribution Detection](https://mlsanity.com/paper/2109.05642/) (2021)
- [DAGs with No Fears: A Closer Look at Continuous Optimization for Learning Bayesian Networks](https://mlsanity.com/paper/2010.09133/) (2020)
- [How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context](https://mlsanity.com/paper/2002.00652/) (2020)
- [Unsuitability of NOTEARS for Causal Graph Discovery](https://mlsanity.com/paper/2104.05441/) (2021)
- [Understanding Why Neural Networks Generalize Well Through GSNR of Parameters](https://mlsanity.com/paper/2001.07384/) (2020)
- [A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models](https://mlsanity.com/paper/2303.18211/) (2023)
- [Standardizing Structural Causal Models](https://mlsanity.com/paper/2406.11601/) (2024)
- [Causal Modeling of Twitter Activity During COVID-19](https://mlsanity.com/paper/2005.07952/) (2020)
- [Evaluation of Induced Expert Knowledge in Causal Structure Learning by NOTEARS](https://mlsanity.com/paper/2301.01817/) (2023)
- [Invariant Structure Learning for Better Generalization and Causal Explainability](https://mlsanity.com/paper/2206.06469/) (2022)
- [Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)](https://mlsanity.com/paper/2309.13681/) (2023)
- [A step towards the applicability of algorithms based on invariant causal learning on observational data](https://mlsanity.com/paper/2304.02286/) (2023)

## Built on by (most cited first, 1 of 1)

- [Quality-Controlled Active Learning via Gaussian Processes for Robust Structure-Property Learning in Autonomous Microscopy](https://mlsanity.com/paper/2603.29135/) (2026)

## Abstract (arXiv)

Improving generalization and achieving highly predictive, robust machine learning models necessitates learning the underlying causal structure of the variables of interest. A prominent and effective method for this is learning invariant predictors across multiple environments. In this work, we introduce a simple yet powerful approach, CGLearn, which relies on the agreement of gradients across various environments. This agreement serves as a powerful indication of reliable features, while disagreement suggests less reliability due to potential differences in underlying causal mechanisms. Our proposed method demonstrates superior performance compared to state-of-the-art methods in both linear and nonlinear settings across various regression and classification tasks. CGLearn shows robust applicability even in the absence of separate environments by exploiting invariance across different subsamples of observational data. Comprehensive experiments on both synthetic and real-world datasets highlight its effectiveness in diverse scenarios. Our findings underscore the importance of leveraging gradient agreement for learning causal invariance, providing a significant step forward in the field of robust machine learning. The source code of the linear and nonlinear implementation of CGLearn is open-source and available at: https://github.com/hasanjawad001/CGLearn.