# Causal Regularization: On the trade-off between in-sample risk and out-of-sample risk guarantees

arXiv: https://arxiv.org/abs/2205.01593 · HTML: https://mlsanity.com/paper/2205.01593/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 May 2022.

## Built on by (most cited first, 2 of 2)

- [Causality-oriented robustness: exploiting general noise interventions](https://mlsanity.com/paper/2307.10299/) (2023)
- [An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation](https://mlsanity.com/paper/2510.25128/) (2025)

## Abstract (arXiv)

Invariant prediction uses the prediction stability of causal relationships across different environments to identify causal variables. Conversely, using causal variables gives prediction guarantees even in out-of-sample data settings. In this paper, we investigate the identification of causal-like models from in-sample data that ensure out-of-sample risk guarantees when predicting a target variable from an arbitrary set of covariates. Ordinary least squares minimizes in-sample risk but offers limited out-of-sample guarantees, while causal models optimize out-of-sample guarantees at the expense of in-sample performance. We introduce a form of textitcausal regularization to balance these properties. In the population setting, higher regularization yields estimators with greater risk stability, albeit with increased in-sample risk. Empirically, however, there is a further trade-off to consider, as finite in-sample data reduced the ability to correctly identify models with high out-of-sample risk guarantees. We show how in such empirical settings the optimal causal regularizer can be found via cross-validation.