mlsanityData to 7 Oct 2026

Design of Experiments for Stochastic Contextual Linear Bandits

arXiv 2107.09912 ↗First posted 21 Jul 2021cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

In the stochastic linear contextual bandit setting there exist several minimax procedures for exploration with policies that are reactive to the data being acquired. In practice, there can be a significant engineering overhead to deploy these algorithms, especially when the dataset is collected in a distributed fashion or when a human in the loop is needed to implement a different policy. Exploring with a single non-reactive policy is beneficial in such cases. Assuming some batch contexts are available, we design a single stochastic policy to collect a good dataset from which a near-optimal policy can be extracted. We present a theoretical analysis as well as numerical experiments on both synthetic and real-world datasets.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 18 of its 33 references that are in the map. First posted on arXiv 21 Jul 2021.

Builds on

18 of 33 references.

References · 18

  1. 2018Is Q-learning Provably Efficient?cited by 485
  2. 2020Reward-Free Exploration for Reinforcement Learningcited by 177
  3. 2019Learning with Good Feature Representations in Bandits and in RL with a Generative Modelcited by 124
  4. 2021Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraintscited by 124
  5. 2014Best-Arm Identification in Linear Banditscited by 98
  6. 2018Practical Contextual Bandits with Regression Oraclescited by 92
  7. 2020On Reward-Free Reinforcement Learning with Linear Function Approximationcited by 89
  8. 2019Sequential Experimental Design for Transductive Linear Banditscited by 88
  9. 2019Provably Efficient Q-Learning with Low Switching Costcited by 77
  10. 2020Optimal Best-arm Identification in Linear Banditscited by 70
  11. 2020Sequential Batch Learning in Finite-Action Linear Contextual Banditscited by 63
  12. 2020Gamification of Pure Exploration for Linear Banditscited by 61
6 more
  1. 2021Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximationcited by 53
  2. 2020Linear Bandits with Limited Adaptivity and Learning Distributional Optimal Designcited by 51
  3. 2019Regret Bounds for Batched Banditscited by 49
  4. 2017Fully adaptive algorithm for pure exploration in linear banditscited by 43
  5. 2020Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimensioncited by 36
  6. 2018Simple Regret Minimization for Contextual Banditscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 6; 2024: 2; 2025: 1; 2026: 3.

18 papers.

Sort:

By area:

  1. 2022Instance-optimal PAC Algorithms for Contextual Banditscited by 16
  2. 2022Safe Exploration for Efficient Policy Evaluation and Comparisoncited by 15
  3. 2021Learning to be Fair: A Consequentialist Approach to Equitable Decision-Makingcited by 11
  4. 2023Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Datacited by 11
  5. 2023Concentrated Differential Privacy for Banditscited by 10
  6. 2021Almost Optimal Batch-Regret Tradeoff for Batch Linear Contextual Banditscited by 9
  7. 2023Proportional Response: Contextual Bandits for Simple and Cumulative Regret Minimizationcited by 7
  8. 2023Multi-task Representation Learning for Pure Exploration in Linear Banditscited by 6
  9. 2024Experiment Planning with Function Approximationcited by 5
  10. 2026Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferencescited by 5
  11. 2023CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Designcited by 4
  12. 2022Efficient Real-world Testing of Causal Decision Making via Bayesian Experimental Design for Contextual Optimisationcited by 3
6 more
  1. 2024Active Few-Shot Fine-Tuningcited by 2
  2. 2022Anonymous Bandits for Multi-User Systemscited by 1
  3. 2026Active Learning for Stochastic Contextual Linear Banditscited by 1
  4. 2026Anytime-valid Optimal Policy Identificationcited by 1
  5. 2023Estimating Optimal Policy Value in General Linear Contextual Banditscited by 0
  6. 2025Riemannian Manifold Learning for Stackelberg Games with Neural Flow Representationscited by 0