mlsanityData to 7 Oct 2026

Bandit Algorithms for Precision Medicine

arXiv 2108.04782 ↗First posted 10 Aug 2021stat.ML cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

The Oxford English Dictionary defines precision medicine as "medical care designed to optimize efficiency or therapeutic benefit for particular groups of patients, especially by using genetic or molecular profiling." It is not an entirely new idea: physicians from ancient times have recognized that medical treatment needs to consider individual variations in patient characteristics. However, the modern precision medicine movement has been enabled by a confluence of events: scientific advances in fields such as genetics and pharmacology, technological advances in mobile devices and wearable sensors, and methodological advances in computing and data sciences. This chapter is about bandit algorithms: an area of data science of special relevance to precision medicine. With their roots in the seminal work of Bellman, Robbins, Lai and others, bandit algorithms have come to occupy a central place in modern data science ( Lattimore and Szepesvari, 2020). Bandit algorithms can be used in any situation where treatment decisions need to be made to optimize some health outcome. Since precision medicine focuses on the use of patient characteristics to guide treatment, contextual bandit algorithms are especially useful since they are designed to take such information into account. The role of bandit algorithms in areas of precision medicine such as mobile health and digital phenotyping has been reviewed before (Tewari and Murphy, 2017; Rabbi et al., 2019). Since these reviews were published, bandit algorithms have continued to find uses in mobile health and several new topics have emerged in the research on bandit algorithms. This chapter is written for quantitative researchers in fields such as statistics, machine learning, and operations research who might be interested in knowing more about the algorithmic and mathematical details of bandit algorithms that have been used in mobile health.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 53 of its 129 references that are in the map. First posted on arXiv 10 Aug 2021.

Builds on

53 of 129 references.

References · 53

  1. 2017A Survey on Multi-Task Learningcited by 838
  2. 2017Counterfactual Fairnesscited by 802
  3. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  4. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  5. 2015The Benefit of Multitask Representation Learningcited by 194
  6. 2018Fairness of Exposure in Rankingscited by 189
  7. 2016Fairness in Learning: Classic and Contextual Banditscited by 167
  8. 2013Bandits with Knapsackscited by 163
  9. 2015Doubly Robust Policy Evaluation and Optimizationcited by 158
  10. 2015Explore no more: Improved high-probability regret bounds for non-stochastic banditscited by 120
  11. 2018Stochastic bandits robust to adversarial corruptionscited by 118
  12. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
38 more
  1. 2016Causal Bandits: Learning Good Interventions via Causal Inferencecited by 100
  2. 2014Online Clustering of Banditscited by 99
  3. 2021Non-stationary Reinforcement Learning without Prior Knowledge: An Optimal Black-box Approachcited by 98
  4. 2019Better Algorithms for Stochastic Bandits with Adversarial Corruptionscited by 91
  5. 2014Bandits with concave rewards and convex knapsackscited by 83
  6. 2015Linear Contextual Bandits with Knapsackscited by 82
  7. 2013Confidence Intervals and Hypothesis Testing for High-Dimensional Regressioncited by 78
  8. 2019Achieving Fairness in the Stochastic Multi-armed Bandit Problemcited by 77
  9. 2018Adversarial Bandits with Knapsackscited by 73
  10. 2019A New Algorithm for Non-stationary Contextual Bandits: Efficient, Optimal, and Parameter-freecited by 71
  11. 2019Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activitycited by 70
  12. 2018Online Learning with an Unknown Fairness Metriccited by 61
  13. 2020High-Dimensional Sparse Linear Banditscited by 61
  14. 2016Conservative Banditscited by 58
  15. 2016Conservative Contextual Linear Banditscited by 56
  16. 2013A Gang of Banditscited by 49
  17. 2018Nearly Optimal Adaptive Procedure with Change Detection for Piecewise-Stationary Banditcited by 49
  18. 2020Low-Rank Generalized Linear Bandit Problemscited by 48
  19. 2017Multi-Task Learning for Contextual Banditscited by 47
  20. 2017Calibrated Fairness in Banditscited by 42
  21. 2021Fairness of Exposure in Stochastic Banditscited by 42
  22. 2017Identifying Best Interventions through Online Importance Samplingcited by 39
  23. 2016A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patientscited by 38
  24. 2020Budgeted and Non-budgeted Causal Banditscited by 38
  25. 2021Causal Bandits with Unknown Graph Structurecited by 37
  26. 2020Power Constrained Banditscited by 30
  27. 2015Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Banditscited by 29
  28. 2020MOTS: Minimax Optimal Thompson Samplingcited by 28
  29. 2020IntelligentPooling: Practical Thompson Sampling for mHealthcited by 24
  30. 2015Thompson Sampling for Budgeted Multi-armed Banditscited by 23
  31. 2020The Elliptical Potential Lemma Revisitedcited by 23
  32. 2021Doubly robust Thompson sampling for linear payoffscited by 21
  33. 2020Conservative Exploration in Reinforcement Learningcited by 20
  34. 2020Improved Algorithms for Conservative Exploration in Banditscited by 16
  35. 2017Budget-Constrained Multi-Armed Bandits with Multiple Playscited by 13
  36. 2020Offline Contextual Multi-armed Bandits for Mobile Health Interventions: A Case Study on Emotion Regulationcited by 13
  37. 2020Achieving User-Side Fairness in Contextual Banditscited by 10
  38. 2021Minimax Off-Policy Evaluation for Multi-Armed Banditscited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 2; 2025: 4; 2026: 2.

12 papers.

  1. 2022Transfer Learning for Contextual Multi-armed Banditscited by 24
  2. 2024Sliding-Window Thompson Sampling for Non-Stationary Settingscited by 15
  3. 2025Single Index Bandits: Generalized Linear Contextual Bandits with Unknown Reward Functionscited by 6
  4. 2021Doubly Robust Interval Estimation for Optimal Policy Evaluation in Online Learningcited by 4
  5. 2024Automated Off-Policy Estimator Selection via Supervised Learningcited by 2
  6. 2025Optimal and Practical Batched Linear Bandit Algorithmcited by 2
  7. 2023Efficient Generalized Low-Rank Tensor Contextual Banditscited by 1
  8. 2025Faster Rates for Private Adversarial Banditscited by 1
  9. 2026Practical and Optimal Algorithm for Linear Contextual Bandits with Rare Parameter Updatescited by 1
  10. 2023Leveraging User-Triggered Supervision in Contextual Banditscited by 0
  11. 2025Non-Linear Model-Based Sequential Decision-Making in Agriculturecited by 0
  12. 2026RIE-Greedy: Regularization-Induced Exploration for Contextual Banditscited by 0