# On Inductive Biases for Heterogeneous Treatment Effect Estimation

arXiv: https://arxiv.org/abs/2106.03765 · HTML: https://mlsanity.com/paper/2106.03765/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 73 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Explicit Inductive Bias for Transfer Learning with Convolutional Networks](https://mlsanity.com/paper/1802.01483/) (2018)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes](https://mlsanity.com/paper/1704.02801/) (2017)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms](https://mlsanity.com/paper/2101.10943/) (2021)
- [Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition](https://mlsanity.com/paper/1707.02641/) (2017)
- [Counterfactual Representation Learning with Balancing Weights](https://mlsanity.com/paper/2010.12618/) (2020)
- [Learning Weighted Representations for Generalization Across Designs](https://mlsanity.com/paper/1802.08598/) (2018)
- [Deep Counterfactual Networks with Propensity-Dropout](https://mlsanity.com/paper/1706.05966/) (2017)
- [Hypothesis Transfer Learning via Transformation Functions](https://mlsanity.com/paper/1612.01020/) (2016)
- [Estimating Average Treatment Effects via Orthogonal Regularization](https://mlsanity.com/paper/2101.08490/) (2021)
- [Counterfactual Predictions under Runtime Confounding](https://mlsanity.com/paper/2006.16916/) (2020)

## Built on by (most cited first, 30 of 75)

- [CausalPFN: Amortized Causal Effect Estimation via In-Context Learning](https://mlsanity.com/paper/2506.07918/) (2025)
- [Combining Observational and Randomized Data for Estimating Heterogeneous Treatment Effects](https://mlsanity.com/paper/2202.12891/) (2022)
- [In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2302.02923/) (2023)
- [Exploring Transformer Backbones for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2202.01336/) (2022)
- [Explicit Feature Interaction-aware Uplift Network for Online Marketing](https://mlsanity.com/paper/2306.00315/) (2023)
- [Conformal Meta-learners for Predictive Inference of Individual Treatment Effects](https://mlsanity.com/paper/2308.14895/) (2023)
- [Transfer Learning on Heterogeneous Feature Spaces for Treatment Effects Estimation](https://mlsanity.com/paper/2210.06183/) (2022)
- [Causal Deep Learning](https://mlsanity.com/paper/2303.02186/) (2023)
- [Bounds on Representation-Induced Confounding Bias for Treatment Effect Estimation](https://mlsanity.com/paper/2311.11321/) (2023)
- [Zero-shot causal learning](https://mlsanity.com/paper/2301.12292/) (2023)
- [Estimating Treatment Effects from Irregular Time Series Observations with Hidden Confounders](https://mlsanity.com/paper/2303.02320/) (2023)
- [DiffPO: A causal diffusion model for learning distributions of potential outcomes](https://mlsanity.com/paper/2410.08924/) (2024)
- [Deconfounding Temporal Autoencoder: Estimating Treatment Effects over Time Using Noisy Proxies](https://mlsanity.com/paper/2112.03013/) (2021)
- [Benchmarking Heterogeneous Treatment Effect Models through the Lens of Interpretability](https://mlsanity.com/paper/2206.08363/) (2022)
- [Accounting For Informative Sampling When Learning to Forecast Treatment Outcomes Over Time](https://mlsanity.com/paper/2306.04255/) (2023)
- [Deep Causal Learning: Representation, Discovery and Inference](https://mlsanity.com/paper/2211.03374/) (2022)
- [Dynamic Inter-treatment Information Sharing for Individualized Treatment Effects Estimation](https://mlsanity.com/paper/2305.15984/) (2023)
- [Meta-Learners for Partially-Identified Treatment Effects Across Multiple Environments](https://mlsanity.com/paper/2406.02464/) (2024)
- [Experimental Standards for Deep Learning in Natural Language Processing Research](https://mlsanity.com/paper/2204.06251/) (2022)
- [Comparison of meta-learners for estimating multi-valued treatment heterogeneous effects](https://mlsanity.com/paper/2205.14714/) (2022)
- [A Survey of Deep Causal Models and Their Industrial Applications](https://mlsanity.com/paper/2209.08860/) (2022)
- [Adversarial De-confounding in Individualised Treatment Effects Estimation](https://mlsanity.com/paper/2210.10530/) (2022)
- [Conformal Convolution and Monte Carlo Meta-learners for Predictive Inference of Individual Treatment Effects](https://mlsanity.com/paper/2402.04906/) (2024)
- [Treatment Effect Estimation for Optimal Decision-Making](https://mlsanity.com/paper/2505.13092/) (2025)
- [Covariate-Balancing-Aware Interpretable Deep Learning models for Treatment Effect Estimation](https://mlsanity.com/paper/2203.03185/) (2022)
- [Understanding the Impact of Competing Events on Heterogeneous Treatment Effect Estimation from Time-to-Event Data](https://mlsanity.com/paper/2302.12718/) (2023)
- [Multi-Treatment Multi-Task Uplift Modeling for Enhancing User Growth](https://mlsanity.com/paper/2408.12803/) (2024)
- [Orthogonal Representation Learning for Estimating Causal Quantities](https://mlsanity.com/paper/2502.04274/) (2025)
- [Heterogeneous Treatment Effect Estimation using machine learning for Healthcare application: tutorial and benchmark](https://mlsanity.com/paper/2109.12769/) (2021)
- [Heterogeneous Treatment Effect with Trained Kernels of the Nadaraya-Watson Regression](https://mlsanity.com/paper/2207.09139/) (2022)

## Abstract (arXiv)

We investigate how to exploit structural similarities of an individual's potential outcomes (POs) under different treatments to obtain better estimates of conditional average treatment effects in finite samples. Especially when it is unknown whether a treatment has an effect at all, it is natural to hypothesize that the POs are similar - yet, some existing strategies for treatment effect estimation employ regularization schemes that implicitly encourage heterogeneity even when it does not exist and fail to fully make use of shared structure. In this paper, we investigate and compare three end-to-end learning strategies to overcome this problem - based on regularization, reparametrization and a flexible multi-task architecture - each encoding inductive bias favoring shared behavior across POs. To build understanding of their relative strengths, we implement all strategies using neural networks and conduct a wide range of semi-synthetic experiments. We observe that all three approaches can lead to substantial improvements upon numerous baselines and gain insight into performance differences across various experimental settings.