On The Statistical Complexity of Offline Decision-Making
Not on the map. Why?
Abstract (arXiv)
We study the statistical complexity of offline decision-making with function approximation, establishing (near) minimax-optimal rates for stochastic contextual bandits and Markov decision processes. The performance limits are captured by the pseudo-dimension of the (value) function class and a new characterization of the behavior policy that strictly subsumes all the previous notions of data coverage in the offline decision-making literature. In addition, we seek to understand the benefits of using offline data in online decision-making and show nearly minimax-optimal rates in a wide range of regimes.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 30 of its 62 references that are in the map. First posted on arXiv 10 Jan 2025.
Builds on
30 of 62 references.
References · 30
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
- 2017Spectrally-normalized margin bounds for neural networkscited by 686
- 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
- 2016Deep Reinforcement Learning for Dialogue Generationcited by 288
- 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
- 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimismcited by 279
- 2021Bellman-consistent Pessimism for Offline Reinforcement Learningcited by 269
- 2020Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oraclescited by 197
- 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
- 2021Pessimistic Model-based Offline Reinforcement Learning under Partial Coveragecited by 164
18 more
- 2022Hybrid RL: Using Both Offline and Online Data Can Make RL Efficientcited by 161
- 2021Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learningcited by 158
- 2020Bypassing the Monster: A Faster and Simpler Optimal Algorithm for Contextual Bandits under Realizabilitycited by 115
- 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 110
- 2018Practical Contextual Bandits with Regression Oraclescited by 93
- 2021Towards Instance-Optimal Offline Reinforcement Learning with Pessimismcited by 78
- 2021Offline Reinforcement Learning: Fundamental Barriers for Value Function Approximationcited by 71
- 2020On the Value of Target Data in Transfer Learningcited by 65
- 2015Fast rates in statistical and online learningcited by 47
- 2019Off-Policy Policy Gradient with State Distribution Correctioncited by 43
- 2022Leveraging Offline Data in Online Reinforcement Learningcited by 42
- 2017Active Learning for Cost-Sensitive Classificationcited by 41
- 2023Optimal Learners for Realizable Regression: PAC Learning and Online Learningcited by 23
- 2022On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximationcited by 22
- 2023Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learningcited by 16
- 2023Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Datacited by 11
- 2024On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling, and Beyondcited by 6
- 2023Efficient Online Learning with Offline Datasets for Infinite Horizon MDPs: A Bayesian Approachcited by 3
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.