# Causal Regularization

arXiv: https://arxiv.org/abs/1906.12179 · HTML: https://mlsanity.com/paper/1906.12179/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2019.

## Built on by (most cited first, 21 of 21)

- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://mlsanity.com/paper/2207.01848/) (2022)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Harnessing Out-Of-Distribution Examples via Augmenting Content and Style](https://mlsanity.com/paper/2207.03162/) (2022)
- [On Causally Disentangled Representations](https://mlsanity.com/paper/2112.05746/) (2021)
- [Matching Learned Causal Effects of Neural Networks with Domain Priors](https://mlsanity.com/paper/2111.12490/) (2021)
- [On the Limitations of General Purpose Domain Generalisation Methods](https://mlsanity.com/paper/2202.00563/) (2022)
- [Causal Forecasting:Generalization Bounds for Autoregressive Models](https://mlsanity.com/paper/2111.09831/) (2021)
- [Beyond Discriminant Patterns: On the Robustness of Decision Rule Ensembles](https://mlsanity.com/paper/2109.10432/) (2021)
- [Sequential Underspecified Instrument Selection for Cause-Effect Estimation](https://mlsanity.com/paper/2302.05684/) (2023)
- [From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions](https://mlsanity.com/paper/2406.18865/) (2024)
- [Treatment Learning Causal Transformer for Noisy Image Classification](https://mlsanity.com/paper/2203.15529/) (2022)
- [Invariant Structure Learning for Better Generalization and Causal Explainability](https://mlsanity.com/paper/2206.06469/) (2022)
- [Do regularization methods for shortcut mitigation work as intended?](https://mlsanity.com/paper/2503.17015/) (2025)
- [Correcting Confounding via Random Selection of Background Variables](https://mlsanity.com/paper/2202.02150/) (2022)
- [Interpolation and Regularization for Causal Learning](https://mlsanity.com/paper/2202.09054/) (2022)
- [Towards Learning and Explaining Indirect Causal Effects in Neural Networks](https://mlsanity.com/paper/2303.13850/) (2023)
- [An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation](https://mlsanity.com/paper/2510.25128/) (2025)
- [A Consistent Estimator for Confounding Strength](https://mlsanity.com/paper/2211.01903/) (2022)
- [Causal Transfer in Medical Image Analysis](https://mlsanity.com/paper/2603.24388/) (2026)
- [Interventional Flow Matching: Prospective Dose-Response Forecasting with Velocity-Field Jacobian Regularization](https://mlsanity.com/paper/2606.29386/) (2026)
- [Structure-Guided Spatiotemporal Attention Graph Neural Network for Traffic Flow Prediction](https://mlsanity.com/paper/2608.14177/) (2026)

## Abstract (arXiv)

I argue that regularizing terms in standard regression methods not only help against overfitting finite data, but sometimes also yield better causal models in the infinite sample regime. I first consider a multi-dimensional variable linearly influencing a target variable with some multi-dimensional unobserved common cause, where the confounding effect can be decreased by keeping the penalizing term in Ridge and Lasso regression even in the population limit. Choosing the size of the penalizing term, is however challenging, because cross validation is pointless. Here it is done by first estimating the strength of confounding via a method proposed earlier, which yielded some reasonable results for simulated and real data. Further, I prove a `causal generalization bound' which states (subject to a particular model of confounding) that the error made by interpreting any non-linear regression as causal model can be bounded from above whenever functions are taken from a not too rich class. In other words, the bound guarantees "generalization" from observational to interventional distributions, which is usually not subject of statistical learning theory (and is only possible due to the underlying symmetries of the confounder model).