# SIRNET: Causal Adaptation of Pretrained Embeddings for Semiparametric Treatment-Effect Estimation Under Surrogacy

arXiv: https://arxiv.org/abs/2610.14011 · HTML: https://mlsanity.com/paper/2610.14011/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 46 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 18 of 18)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [On Inductive Biases for Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2106.03765/) (2021)
- [Perfect Match: A Simple Method for Learning Representations For Counterfactual Inference With Neural Networks](https://mlsanity.com/paper/1810.00656/) (2018)
- [Causal Effect Inference for Structured Treatments](https://mlsanity.com/paper/2106.01939/) (2021)
- [On the role of surrogates in the efficient estimation of treatment effects with limited outcome data](https://mlsanity.com/paper/2003.12408/) (2020)
- [Long-Term Effect Estimation with Surrogate Representation](https://mlsanity.com/paper/2008.08236/) (2020)
- [Long-term Causal Effects Estimation via Latent Surrogates Representation Learning](https://mlsanity.com/paper/2208.04589/) (2022)
- [Estimating the Long-Term Effects of Novel Treatments](https://mlsanity.com/paper/2103.08390/) (2021)
- [Leveraging semantic similarity for experimentation with AI-generated treatments](https://mlsanity.com/paper/2510.21119/) (2025)
- [TERRA: A Transformer-Enabled Recursive R-learner for Longitudinal Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2510.22407/) (2025)
- [DeepBlip: Estimating Conditional Average Treatment Effects Over Time](https://mlsanity.com/paper/2511.14545/) (2025)
- [Experimentation Accelerator: Interpretable Insights and Creative Recommendations for A/B Testing with Content-Aware ranking](https://mlsanity.com/paper/2602.13852/) (2026)

## Abstract (arXiv)

Modern experiments increasingly produce rich post-treatment objects---such as text, images, and video---that may provide early surrogate signals for outcomes that are delayed, expensive, or observed only for a subset of participants. A pretrained encoder can turn such an object into a strong generic embedding, but the embedding is not trained for the estimation tasks required by a causal estimator and may emphasize information irrelevant to the causal target. We propose SIRNET, a Surrogate-Informed Representation adaptation framework for neural-NETwork-based causal estimation in the presence of unstructured post-treatment surrogates. Starting from pretrained surrogate embeddings, SIRNET learns a shared adapter jointly for outcome prediction, treatment propensity, and arm-specific surrogate marginalization, beyond the generic pretraining objective. The resulting covariate and surrogate representations provide a common interface for these nuisance tasks. The final treatment-effect estimate and confidence interval are obtained from cross-fitted orthogonal scores that attain the semiparametric efficiency bound under the stated nuisance-rate conditions. We also validate SIRNET's performance in synthetic and semisynthetic experiments, as well as the Anti-Amyloid in Asymptomatic Alzheimer's Disease (A4) trial, using frozen pretrained brain-MRI embeddings as surrogates for a long-term outcome.