# Transformer-Based Spatial-Temporal Counterfactual Outcomes Estimation

arXiv: https://arxiv.org/abs/2506.21154 · HTML: https://mlsanity.com/paper/2506.21154/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 45 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Builds on (most cited first, 15 of 15)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations](https://mlsanity.com/paper/2002.04083/) (2020)
- [Causal Transformer for Estimating Counterfactual Outcomes](https://mlsanity.com/paper/2204.07258/) (2022)
- [Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders](https://mlsanity.com/paper/1902.00450/) (2019)
- [Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations](https://mlsanity.com/paper/2206.08311/) (2022)
- [From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering](https://mlsanity.com/paper/2205.14895/) (2022)
- [Neural Point Process for Learning Spatiotemporal Event Dynamics](https://mlsanity.com/paper/2112.06351/) (2021)
- [Estimating Individual Treatment Effects with Time-Varying Confounders](https://mlsanity.com/paper/2008.13620/) (2020)
- [Model Predictive Task Sampling for Efficient and Robust Adaptation](https://mlsanity.com/paper/2501.11039/) (2025)
- [Counterfactual Generative Models for Time-Varying Treatments](https://mlsanity.com/paper/2305.15742/) (2023)
- [Causal Inference with Spatio-temporal Data: Estimating the Effects of Airstrikes on Insurgent Violence in Iraq](https://mlsanity.com/paper/2003.13555/) (2020)
- [G-Net: A Deep Learning Approach to G-computation for Counterfactual Outcome Prediction Under Dynamic Treatment Regimes](https://mlsanity.com/paper/2003.10551/) (2020)
- [Causal Contrastive Learning for Counterfactual Regression Over Time](https://mlsanity.com/paper/2406.00535/) (2024)
- [Bayesian method for causal inference in spatially-correlated multivariate time series](https://mlsanity.com/paper/1801.06282/) (2018)

## Built on by (most cited first, 4 of 4)

- [Correlation or Causation: Analyzing the Causal Structures of LLM and LRM Reasoning Process](https://mlsanity.com/paper/2509.17380/) (2025)
- [SpaCellAgent: A Self-Evolving LLM-Based Multi-Agent Framework for Trajectory Analysis](https://mlsanity.com/paper/2607.07467/) (2026)
- [PERO: Efficient Robust Post-Training Foundation Models for Encrypted Traffic Classification](https://mlsanity.com/paper/2608.15504/) (2026)
- [Bridging Risk Approximation Gaps in Model Predictive Task Sampling via In-Context Modeling](https://mlsanity.com/paper/2610.14913/) (2026)

## Abstract (arXiv)

The real world naturally has dimensions of time and space. Therefore, estimating the counterfactual outcomes with spatial-temporal attributes is a crucial problem. However, previous methods are based on classical statistical models, which still have limitations in performance and generalization. This paper proposes a novel framework for estimating counterfactual outcomes with spatial-temporal attributes using the Transformer, exhibiting stronger estimation ability. Under mild assumptions, the proposed estimator within this framework is consistent and asymptotically normal. To validate the effectiveness of our approach, we conduct simulation experiments and real data experiments. Simulation experiments show that our estimator has a stronger estimation capability than baseline methods. Real data experiments provide a valuable conclusion to the causal effect of conflicts on forest loss in Colombia. The source code is available at https://github.com/lihe-maxsize/DeppSTCI_Release_Version-master.