mlsanityData to 7 Oct 2026

Contextual Information-Directed Sampling

arXiv 2205.10895 ↗First posted 22 May 2022cs.LG math.ST stat.MLcited by 16

Not on the map. Why?

Abstract (arXiv)

Information-directed sampling (IDS) has recently demonstrated its potential as a data-efficient reinforcement learning algorithm. However, it is still unclear what is the right form of information ratio to optimize when contextual information is available. We investigate the IDS design through two contextual bandit problems: contextual bandits with graph feedback and sparse linear contextual bandits. We provably demonstrate the advantage of contextual IDS over conditional IDS and emphasize the importance of considering the context distribution. The main message is that an intelligent agent should invest more on the actions that are beneficial for the future unseen contexts while the conditional IDS can be myopic. We further propose a computationally-efficient version of contextual IDS based on Actor-Critic and evaluate it empirically on a neural network contextual bandit.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 22 May 2022.

Builds on

25 of 37 references.

References · 25

  1. 2013Learning to Optimize Via Posterior Samplingcited by 253
  2. 2014Taming the Monster: A Fast and Simple Algorithm for Contextual Banditscited by 200
  3. 2020Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oraclescited by 195
  4. 2014Learning to Optimize via Information-Directed Samplingcited by 168
  5. 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 114
  6. 2015Online Learning with Feedback Graphs: Beyond Banditscited by 79
  7. 2018Information Directed Sampling and Bandits with Heteroscedastic Noisecited by 75
  8. 2014Nonstochastic Multi-Armed Bandits with Graph-Structured Feedbackcited by 61
  9. 2020High-Dimensional Sparse Linear Banditscited by 61
  10. 2021Reinforcement Learning, Bit by Bitcited by 59
  11. 2019Doubly-Robust Lasso Banditcited by 49
  12. 2020Sparsity-Agnostic Lasso Banditcited by 44
13 more
  1. 2020Information Directed Sampling for Linear Partial Monitoringcited by 39
  2. 2020Mirror Descent and the Information Ratiocited by 39
  3. 2020Asymptotically Optimal Information-Directed Samplingcited by 33
  4. 2020Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Banditscited by 28
  5. 2020Online Sparse Reinforcement Learningcited by 25
  6. 2017Information Directed Sampling for Stochastic Bandits with Graph Feedbackcited by 24
  7. 2021Information Directed Sampling for Sparse Linear Banditscited by 20
  8. 2020Structured Linear Contextual Bandits: A Sharp and Geometric Smoothed Analysiscited by 15
  9. 2021The Value of Information When Deciding What to Learncited by 15
  10. 2020Nearly Dimension-Independent Sparse Linear Bandit over Small Action Spaces via Best Subset Selectioncited by 12
  11. 2017Thompson Sampling For Stochastic Bandits with Graph Feedbackcited by 9
  12. 2018Analysis of Thompson Sampling for Graphical Bandits Without the Graphscited by 7
  13. 2020Reinforcement Learning with Feedback Graphscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 4; 2025: 3; 2026: 1.

16 papers.

Sort:

By area:

  1. 2022Regret Bounds for Information-Directed Reinforcement Learningcited by 24
  2. 2022Lifting the Information Ratio: An Information-Theoretic Analysis of Thompson Sampling for Contextual Banditscited by 19
  3. 2022Thompson Sampling for High-Dimensional Sparse Linear Contextual Banditscited by 16
  4. 2023Linear Partial Monitoring for Sequential Decision-Making: Algorithms, Regret Bounds and Applicationscited by 11
  5. 2023An Information-Theoretic Analysis of Nonstationary Bandit Learningcited by 11
  6. 2023A Definition of Non-Stationary Banditscited by 9
  7. 2023Improved Bayesian Regret Bounds for Thompson Sampling in Reinforcement Learningcited by 7
  8. 2024Optimistic Information Directed Samplingcited by 7
  9. 2025Adaptive Context Length Optimization with Low-Frequency Truncation for Multi-Agent Reinforcement Learningcited by 6
  10. 2024Stochastic contextual bandits with graph feedback: from independence number to MAS numbercited by 5
  11. 2025CAReDiO: Cultural Alignment via Representativeness and Distinctiveness Guided Data Optimizationcited by 5
  12. 2023Bayesian Reinforcement Learning with Limited Cognitive Loadcited by 4
4 more
  1. 2025Sample-Efficient Reinforcement Learning from Human Feedback via Information-Directed Samplingcited by 4
  2. 2024Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysiscited by 2
  3. 2024Reinforcement Learning with Intrinsically Motivated Feedback Graph for Lost-sales Inventory Controlcited by 1
  4. 2026Future Information-Directed Sampling for Bayesian Nonstationary Banditscited by 0