mlsanityData to 14 Oct 2026

Logarithmic Neyman Regret for Adaptive Estimation of the Average Treatment Effect

arXiv 2411.14341 ↗First posted 21 Nov 2024stat.ML cs.LGcited by 4

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Estimation of the Average Treatment Effect (ATE) is a core problem in causal inference with strong connections to Off-Policy Evaluation in Reinforcement Learning. This paper considers the problem of adaptively selecting the treatment allocation probability in order to improve estimation of the ATE. The majority of prior work on adaptive ATE estimation focus on asymptotic guarantees, and in turn overlooks important practical considerations such as the difficulty of learning the optimal treatment allocation as well as hyper-parameter selection. Existing non-asymptotic methods are limited by poor empirical performance and exponential scaling of the Neyman regret with respect to problem parameters. In order to address these gaps, we propose and analyze the Clipped Second Moment Tracking (ClipSMT) algorithm, a variant of an existing algorithm with strong asymptotic optimality guarantees, and provide finite sample bounds on its Neyman regret. Our analysis shows that ClipSMT achieves exponential improvements in Neyman regret on two fronts: improving the dependence on T from O(√T) to O(log T), as well as reducing the exponential dependence on problem parameters to a polynomial dependence. Finally, we conclude with simulations which show the marked improvement of ClipSMT over existing approaches.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 22 of its 42 references that are in the map. First posted on arXiv 21 Nov 2024.

Builds on

22 of 42 references.

References · 22

  1. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  2. 2018Time-uniform, nonparametric, nonasymptotic confidence sequencescited by 229
  3. 2020Estimating means of bounded random variables by bettingcited by 169
  4. 2020Minimax-Optimal Off-Policy Evaluation with Linear Function Approximationcited by 120
  5. 2019Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processescited by 119
  6. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  7. 2019Confidence Intervals for Policy Evaluation in Adaptive Experimentscited by 95
  8. 2020Inference for Batched Banditscited by 63
  9. 2016On Explore-Then-Commit Strategiescited by 47
  10. 2021Statistical Inference with M-Estimators on Adaptively Collected Datacited by 40
  11. 2022Anytime-valid off-policy inference for contextual banditscited by 37
  12. 2021Off-Policy Risk Assessment in Contextual Banditscited by 29
10 more
  1. 2017Data-Efficient Policy Evaluation Through Behavior Policy Searchcited by 26
  2. 2021Active Offline Policy Selectioncited by 26
  3. 2020Efficient Adaptive Experimental Design for Average Treatment Effect Estimationcited by 23
  4. 2023Semiparametric Efficient Inference in Adaptive Experimentscited by 23
  5. 2023Clip-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experimentscited by 16
  6. 2021Off-policy Confidence Sequencescited by 15
  7. 2023Instance-Optimality in Interactive Decision Making: Toward a Non-Asymptotic Theorycited by 12
  8. 2023Optimal Treatment Allocation for Efficient Policy Evaluation in Sequential Decision Makingcited by 11
  9. 2021Minimax Off-Policy Evaluation for Multi-Armed Banditscited by 9
  10. 2022Off-Policy Risk Assessment in Markov Decision Processescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Efficient Adaptive Experimentation with Noncompliancecited by 2
  2. 2025Minimax and Bayes Optimal Adaptive Experimental Design for Treatment Choicecited by 0
  3. 2026Adaptive Experimentation for Censored Survival Outcomescited by 0
  4. 2026Confidence Horizonscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Fast Rates for the Regret of Offline Reinforcement Learning
  2. 2025Logarithmic Regret for Online KL-Regularized Reinforcement Learning
  3. 2026Variance-Adaptive Optimal Algorithm for Reinforcement Learning with Multinomial Logit Function Approximation
  4. 2026Finite Constant Frontiers and Auditable Regret Certificates for Average-Reward Reinforcement Learning
  5. 2025Regret Analysis of Average-Reward Unichain MDPs via an Actor-Critic Approach
  6. 2024Test-Time Regret Minimization in Meta Reinforcement Learning
  7. 2026Model-Based Reinforcement Learning with Double Oracle Efficiency in Policy Optimization and Offline Estimation
  8. 2024The Perils of Optimizing Learned Reward Functions: Low Training Error Does Not Guarantee Low Regret