# Estimating counterfactual treatment outcomes over time in complex multiagent scenarios

arXiv: https://arxiv.org/abs/2206.01900 · HTML: https://mlsanity.com/paper/2206.01900/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 90 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Graph WaveNet for Deep Spatial-Temporal Graph Modeling](https://mlsanity.com/paper/1906.00121/) (2019)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Learning Representations for Counterfactual Inference](https://mlsanity.com/paper/1605.03661/) (2016)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Estimating Counterfactual Treatment Outcomes over Time Through Adversarially Balanced Representations](https://mlsanity.com/paper/2002.04083/) (2020)
- [Sequential Neural Models with Stochastic Layers](https://mlsanity.com/paper/1605.07571/) (2016)
- [Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning](https://mlsanity.com/paper/1811.02307/) (2018)
- [Actions Speak Louder Than Goals: Valuing Player Actions in Soccer](https://mlsanity.com/paper/1802.07127/) (2018)
- [Time Series Deconfounder: Estimating Treatment Effects over Time in the Presence of Hidden Confounders](https://mlsanity.com/paper/1902.00450/) (2019)
- [Learning Physical Intuition of Block Towers by Example](https://mlsanity.com/paper/1603.01312/) (2016)
- [Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling](https://mlsanity.com/paper/2102.13156/) (2021)
- [Z-Forcing: Training Stochastic Recurrent Networks](https://mlsanity.com/paper/1711.05411/) (2017)
- [Generating Multi-Agent Trajectories using Programmatic Weak Supervision](https://mlsanity.com/paper/1803.07612/) (2018)
- [Human Trajectory Prediction via Counterfactual Analysis](https://mlsanity.com/paper/2107.14202/) (2021)
- [Deep Reinforcement Learning in Ice Hockey for Context-Aware Player Evaluation](https://mlsanity.com/paper/1805.11088/) (2018)
- [CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning](https://mlsanity.com/paper/2106.13364/) (2021)
- [Estimating Individual Treatment Effects with Time-Varying Confounders](https://mlsanity.com/paper/2008.13620/) (2020)
- [Evaluation of soccer team defense based on prediction models of ball recovery and being attacked: A pilot study](https://mlsanity.com/paper/2103.09627/) (2021)
- [A Bayesian Nonparametric Approach for Estimating Individualized Treatment-Response Curves](https://mlsanity.com/paper/1608.05182/) (2016)
- [Decentralized policy learning with partial observation and mechanical constraints for multiperson modeling](https://mlsanity.com/paper/2007.03155/) (2020)
- [Learning interaction rules from multi-animal trajectories via augmented behavioral models](https://mlsanity.com/paper/2107.05326/) (2021)
- [Physically-interpretable classification of biological network dynamics for complex collective motions](https://mlsanity.com/paper/1905.04859/) (2019)
- [Grab the Reins of Crowds: Estimating the Effects of Crowd Movement Guidance Using Causal Inference](https://mlsanity.com/paper/2102.03980/) (2021)
- [Conservative Policy Construction Using Variational Autoencoders for Logged Data with Missing Values](https://mlsanity.com/paper/2109.03747/) (2021)

## Built on by (most cited first, 12 of 12)

- [Counterfactual Learning on Graphs: A Survey](https://mlsanity.com/paper/2304.01391/) (2023)
- [Counterfactual Generative Models for Time-Varying Treatments](https://mlsanity.com/paper/2305.15742/) (2023)
- [Action valuation of on- and off-ball soccer players based on multi-agent deep reinforcement learning](https://mlsanity.com/paper/2305.17886/) (2023)
- [Adaptive action supervision in reinforcement learning from real-world multi-agent demonstrations](https://mlsanity.com/paper/2305.13030/) (2023)
- [OpenSTARLab: Open Approach for Spatio-Temporal Agent Data Analysis in Soccer](https://mlsanity.com/paper/2502.02785/) (2025)
- [Mathematical models for off-ball scoring prediction in basketball](https://mlsanity.com/paper/2406.08749/) (2024)
- [TrackID3x3: A Dataset and Algorithm for Multi-Player Tracking with Identification and Pose Estimation in 3x3 Basketball Full-court Videos](https://mlsanity.com/paper/2503.18282/) (2025)
- [Evaluating Movement Initiation Timing in Ultimate Frisbee via Temporal Counterfactuals](https://mlsanity.com/paper/2508.17611/) (2025)
- [Expandable Decision-Making States for Multi-Agent Deep Reinforcement Learning in Soccer Tactical Analysis](https://mlsanity.com/paper/2510.00480/) (2025)
- [Spatio-Temporal Graphical Counterfactuals: An Overview](https://mlsanity.com/paper/2407.01875/) (2024)
- [Data-driven simulator of multi-animal behavior with unknown dynamics via offline and online reinforcement learning](https://mlsanity.com/paper/2510.10451/) (2025)
- [Strategic Multi-Agent Learning for Interpretable Action Valuation of All Players in Football](https://mlsanity.com/paper/2610.05961/) (2026)

## Abstract (arXiv)

Evaluation of intervention in a multiagent system, e.g., when humans should intervene in autonomous driving systems and when a player should pass to teammates for a good shot, is challenging in various engineering and scientific fields. Estimating the individual treatment effect (ITE) using counterfactual long-term prediction is practical to evaluate such interventions. However, most of the conventional frameworks did not consider the time-varying complex structure of multiagent relationships and covariate counterfactual prediction. This may lead to erroneous assessments of ITE and difficulty in interpretation. Here we propose an interpretable, counterfactual recurrent network in multiagent systems to estimate the effect of the intervention. Our model leverages graph variational recurrent neural networks and theory-based computation with domain knowledge for the ITE estimation framework based on long-term prediction of multiagent covariates and outcomes, which can confirm the circumstances under which the intervention is effective. On simulated models of an automated vehicle and biological agents with time-varying confounders, we show that our methods achieved lower estimation errors in counterfactual covariates and the most effective treatment timing than the baselines. Furthermore, using real basketball data, our methods performed realistic counterfactual predictions and evaluated the counterfactual passes in shot scenarios.