mlsanityData to 7 Oct 2026

Deep Spectral Q-learning with Application to Mobile Health

arXiv 2301.00927 ↗First posted 3 Jan 2023stat.ML cs.LG stat.APcited by 0

Not on the map. Why?

Abstract (arXiv)

Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time-varying covariates. In mobile health applications, these covariates are typically collected at different frequencies over a long time horizon. In this paper, we propose a deep spectral Q-learning algorithm, which integrates principal component analysis (PCA) with deep Q-learning to handle the mixed frequency data. In theory, we prove that the mean return under the estimated optimal policy converges to that under the optimal one and establish its rate of convergence. The usefulness of our proposal is further illustrated via simulations and an application to a diabetes dataset.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 97 references that are in the map. First posted on arXiv 3 Jan 2023.

Builds on

40 of 97 references.

References · 40

  1. 2015Trust Region Policy Optimizationcited by 2,503
  2. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  7. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  8. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  9. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  10. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  11. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 285
  12. 2019Batch Policy Learning under Constraintscited by 242
28 more
  1. 2019A Theoretical Analysis of Deep Q-Learningcited by 212
  2. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  3. 2020Breaking the Sample Size Barrier in Model-Based Reinforcement Learning with a Generative Modelcited by 108
  4. 2019Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learningcited by 75
  5. 2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
  6. 2016Estimating Dynamic Treatment Regimes in Mobile Health Using V-learningcited by 65
  7. 2020Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settingscited by 65
  8. 2022A Survey on Model-based Reinforcement Learningcited by 65
  9. 2021Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiencycited by 54
  10. 2019Learning When-to-Treat Policiescited by 49
  11. 2019Off-Policy Estimation of Long-Term Average Outcomes with Applications to Mobile Healthcited by 42
  12. 2020Bidirectional Model-based Policy Optimizationcited by 38
  13. 2021Deeply-Debiased Off-Policy Interval Estimationcited by 36
  14. 2020Does the Markov Decision Process Fit the Data: Testing for the Markov Property in Sequential Decision Makingcited by 30
  15. 2020Dynamic Causal Effects Evaluation in A/B Testing with a Reinforcement Learning Frameworkcited by 29
  16. 2020Learning Optimal Distributionally Robust Individualized Treatment Rulescited by 28
  17. 2021Online Bootstrap Inference For Policy Evaluation in Reinforcement Learningcited by 26
  18. 2021Estimating Optimal Infinite Horizon Dynamic Treatment Regimes via pT-Learningcited by 26
  19. 2022Addressing Optimism Bias in Sequence Modeling for Reinforcement Learningcited by 22
  20. 2022Future-Dependent Value-Based Off-Policy Evaluation in POMDPscited by 20
  21. 2016Interpretable Dynamic Treatment Regimescited by 17
  22. 2020Causal Effect Estimation and Optimal Dose Suggestions in Mobile Healthcited by 8
  23. 2022Reinforcement Learning with Heterogeneous Data: Estimation and Inferencecited by 7
  24. 2020Latent-state models for precision medicinecited by 6
  25. 2015Using PCA to Efficiently Represent State Spacescited by 5
  26. 2022Dynamic Matching Bandit For Two-Sided Online Marketscited by 5
  27. 2017Sufficient Markov Decision Processes with Alternating Deep Neural Networkscited by 3
  28. 2015Optimal eigen expansions and uniform boundscited by 1

Built on by

Not cited yet.