mlsanityData to 7 Oct 2026

Policy Evaluation for Temporal and/or Spatial Dependent Experiments

arXiv 2202.10887 ↗First posted 22 Feb 2022stat.ME cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

The aim of this paper is to establish a causal link between the policies implemented by technology companies and the outcomes they yield within intricate temporal and/or spatial dependent experiments. We propose a novel temporal/spatio-temporal Varying Coefficient Decision Process (VCDP) model, capable of effectively capturing the evolving treatment effects in situations characterized by temporal and/or spatial dependence. Our methodology encompasses the decomposition of the Average Treatment Effect (ATE) into the Direct Effect (DE) and the Indirect Effect (IE). We subsequently devise comprehensive procedures for estimating and making inferences about both DE and IE. Additionally, we provide a rigorous analysis of the statistical properties of these procedures, such as asymptotic power. To substantiate the effectiveness of our approach, we carry out extensive simulations and real data analyses.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 31 of its 67 references that are in the map. First posted on arXiv 22 Feb 2022.

Builds on

31 of 67 references.

References · 31

  1. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  2. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  3. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  4. 2018Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimationcited by 226
  5. 2019Deep Network Approximation Characterized by Number of Neuronscited by 120
  6. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 118
  7. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  8. 2019Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learningcited by 75
  9. 2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
  10. 2016Estimating Dynamic Treatment Regimes in Mobile Health Using V-learningcited by 65
  11. 2020Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settingscited by 65
  12. 2021A Deep Value-network Based Approach for Multi-Driver Order Dispatchingcited by 45
19 more
  1. 2019Off-Policy Estimation of Long-Term Average Outcomes with Applications to Mobile Healthcited by 42
  2. 2020Experimental Design in Two-Sided Platforms: An Analysis of Biascited by 39
  3. 2021Deeply-Debiased Off-Policy Interval Estimationcited by 36
  4. 2021Reinforcement Learning for Ridesharing: An Extended Surveycited by 35
  5. 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
  6. 2016Assessing Time-Varying Causal Effect Moderation in Mobile Healthcited by 27
  7. 2020Double/Debiased Machine Learning for Dynamic Treatment Effects via g-Estimationcited by 27
  8. 2021Off-Policy Evaluation in Partially Observed Markov Decision Processes under Sequential Ignorabilitycited by 24
  9. 2017Time series experiments and causal estimands: exact randomization tests and tradingcited by 21
  10. 2014Designing and Deploying Online Field Experimentscited by 17
  11. 2019Experimenting in Equilibriumcited by 16
  12. 2021Treatment Effects in Market Equilibriumcited by 16
  13. 2020A review of spatial causal inference methods for environmental and epidemiological applicationscited by 12
  14. 2021Graph-Based Equilibrium Metrics for Dynamic Supply-Demand Systems with Applications to Ride-sourcing Platformscited by 11
  15. 2013Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experimentcited by 10
  16. 2019A Graph-Theoretic Approach to Randomization Tests of Causal Effects Under General Interferencecited by 7
  17. 2020Causal Inference for Spatial Treatmentscited by 5
  18. 2019Driver Surge Pricingcited by 4
  19. 2022A Multi-Agent Reinforcement Learning Framework for Off-Policy Evaluation in Two-sided Marketscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2024Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experimentscited by 6
  2. 2024Combining Experimental and Historical Data for Policy Evaluationcited by 6
  3. 2025Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approachcited by 5
  4. 2024Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferencescited by 2
  5. 2025A Two-armed Bandit Framework for A/B Testingcited by 2
  6. 2026Designing Time Series Experiments in A/B Testing with Transformer Reinforcement Learningcited by 1
  7. 2025Generalized Fitted Q-Iteration with Clustered Datacited by 0
  8. 2026Robust Sequential Experimental Design for A/B Testingcited by 0