# Longitudinal Targeted Minimum Loss-based Estimation with Temporal-Difference Heterogeneous Transformer

arXiv: https://arxiv.org/abs/2404.04399 · HTML: https://mlsanity.com/paper/2404.04399/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 41 references that are in the map.
- First posted on arXiv 5 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Doubly Robust Off-policy Value Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1511.03722/) (2015)
- [Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations](https://mlsanity.com/paper/2002.04083/) (2020)
- [More Robust Doubly Robust Off-policy Evaluation](https://mlsanity.com/paper/1802.03493/) (2018)
- [Semiparametric doubly robust targeted double machine learning: a review](https://mlsanity.com/paper/2203.06469/) (2022)
- [Causal Transformer for Estimating Counterfactual Outcomes](https://mlsanity.com/paper/2204.07258/) (2022)
- [Estimating average causal effects from patient trajectories](https://mlsanity.com/paper/2203.01228/) (2022)
- [High-Dimensional Survival Analysis: Methods and Applications](https://mlsanity.com/paper/2205.02948/) (2022)
- [Debiased Off-Policy Evaluation for Recommendation Systems](https://mlsanity.com/paper/2002.08536/) (2020)
- [A Causal Roadmap for Generating High-Quality Real-World Evidence](https://mlsanity.com/paper/2305.06850/) (2023)

## Built on by (most cited first, 10 of 10)

- [IGC-Net for conditional average potential outcome estimation over time](https://mlsanity.com/paper/2405.21012/) (2024)
- [Estimating the treatment effect over time under general interference through deep learner integrated TMLE](https://mlsanity.com/paper/2412.04799/) (2024)
- [Overlap-weighted orthogonal meta-learner for treatment effect estimation over time](https://mlsanity.com/paper/2510.19643/) (2025)
- [Frugal, Flexible, Faithful: Causal Data Simulation via Frengression](https://mlsanity.com/paper/2508.01018/) (2025)
- [DeepBlip: Estimating Conditional Average Treatment Effects Over Time](https://mlsanity.com/paper/2511.14545/) (2025)
- [Efficient Targeted Maximum Likelihood Estimators for Two-Phase Design Problems](https://mlsanity.com/paper/2602.24131/) (2026)
- [Sequential Treatment Effect Estimation with Unmeasured Confounders](https://mlsanity.com/paper/2505.09113/) (2025)
- [Targeted Deep Architectures: A TMLE-Based Framework for Robust Causal Inference in Neural Networks](https://mlsanity.com/paper/2507.12435/) (2025)
- [Deep Doubly Debiased Longitudinal Effect Estimation with ICE G-Computation](https://mlsanity.com/paper/2602.12379/) (2026)
- [Smooth Multi-Policy Causal Effect Estimation in Longitudinal Settings](https://mlsanity.com/paper/2605.14284/) (2026)

## Abstract (arXiv)

We propose Deep Longitudinal Targeted Minimum Loss-based Estimation (Deep LTMLE), a novel approach to estimate the counterfactual mean of outcome under dynamic treatment policies in longitudinal problem settings. Our approach utilizes a transformer architecture with heterogeneous type embedding trained using temporal-difference learning. After obtaining an initial estimate using the transformer, following the targeted minimum loss-based likelihood estimation (TMLE) framework, we statistically corrected for the bias commonly associated with machine learning algorithms. Furthermore, our method also facilitates statistical inference by enabling the provision of 95% confidence intervals grounded in asymptotic statistical theory. Simulation results demonstrate our method's superior performance over existing approaches, particularly in complex, long time-horizon scenarios. It remains effective in small-sample, short-duration contexts, matching the performance of asymptotically efficient estimators. To demonstrate our method in practice, we applied our method to estimate counterfactual mean outcomes for standard versus intensive blood pressure management strategies in a real-world cardiovascular epidemiology cohort study.