# From Text to Treatment Effects: A Meta-Learning Approach to Handling Text-Based Confounding

arXiv: https://arxiv.org/abs/2409.15503 · HTML: https://mlsanity.com/paper/2409.15503/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 16 of 16)

- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Bayesian Inference of Individualized Treatment Effects using Multi-task Gaussian Processes](https://mlsanity.com/paper/1704.02801/) (2017)
- [Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms](https://mlsanity.com/paper/2101.10943/) (2021)
- [On Inductive Biases for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2106.03765/) (2021)
- [Using Wasserstein Generative Adversarial Networks for the Design of Monte Carlo Simulations](https://mlsanity.com/paper/1909.02210/) (2019)
- [Conformal Meta-learners for Predictive Inference of Individual Treatment Effects](https://mlsanity.com/paper/2308.14895/) (2023)
- [BioLORD: Learning Ontological Representations from Definitions (for Biomedical Concepts and their Textual Descriptions)](https://mlsanity.com/paper/2210.11892/) (2022)
- [Bounds on Representation-Induced Confounding Bias for Treatment Effect Estimation](https://mlsanity.com/paper/2311.11321/) (2023)
- [Model-agnostic meta-learners for estimating heterogeneous treatment effects over time](https://mlsanity.com/paper/2407.05287/) (2024)
- [On Weighted Orthogonal Learners for Heterogeneous Treatment Effects](https://mlsanity.com/paper/2303.12687/) (2023)
- [Conformal Convolution and Monte Carlo Meta-learners for Predictive Inference of Individual Treatment Effects](https://mlsanity.com/paper/2402.04906/) (2024)
- [SimSUM: Simulated Benchmark with Structured and Unstructured Medical Records](https://mlsanity.com/paper/2409.08936/) (2024)

## Abstract (arXiv)

One of the central goals of causal machine learning is the accurate estimation of heterogeneous treatment effects from observational data. In recent years, meta-learning has emerged as a flexible, model-agnostic paradigm for estimating conditional average treatment effects (CATE) using any supervised model. This paper examines the performance of meta-learners when the confounding variables are expressed in text. Through synthetic data experiments, we show that learners using pre-trained text representations of confounders, in addition to tabular background variables, achieve improved CATE estimates compared to those relying solely on the tabular variables, particularly when sufficient data is available. However, due to the entangled nature of the text embeddings, these models do not fully match the performance of meta-learners with perfect confounder knowledge. These findings highlight both the potential and the limitations of pre-trained text representations for causal inference and open up interesting avenues for future research.