mlsanityData to 7 Oct 2026

On Practical Reinforcement Learning: Provable Robustness, Scalability, and Statistical Efficiency

arXiv 2203.01758 ↗First posted 3 Mar 2022cs.LG cs.AI stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

This thesis rigorously studies fundamental reinforcement learning (RL) methods in modern practical considerations, including robust RL, distributional RL, and offline RL with neural function approximation. The thesis first prepares the readers with an overall overview of RL and key technical background in statistics and optimization. In each of the settings, the thesis motivates the problems to be studied, reviews the current literature, provides computationally efficient algorithms with provable efficiency guarantees, and concludes with future research directions. The thesis makes fundamental contributions to the three settings above, both algorithmically, theoretically, and empirically, while staying relevant to practical considerations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 172 references that are in the map. First posted on arXiv 3 Mar 2022.

Builds on

36 of 172 references.

Closest ancestors · 5

Same direction, newest first

  1. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization27 Nov
  2. 2021Risk Bounds and Rademacher Complexity in Batch Reinforcement Learning25 Mar
  3. 2021Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism22 Mar
  4. 2020Is Pessimism Provably Efficient for Offline RL?30 Dec
  5. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May

Other references · 31

  1. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  2. 2016OpenAI Gymcited by 1,684
  3. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  4. 2018A Tutorial on Bayesian Optimizationcited by 734
  5. 2018On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transportcited by 422
  6. 2015An Introduction to Matrix Concentration Inequalitiescited by 391
  7. 2021Deep learning: a statistical viewpointcited by 239
  8. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  9. 2019A Theoretical Analysis of Deep Q-Learningcited by 212
  10. 2019AlgaeDICE: Policy Gradient from Arbitrary Experiencecited by 187
  11. 2019Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learningcited by 169
  12. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 160
19 more
  1. 2018More Robust Doubly Robust Off-policy Evaluationcited by 141
  2. 2013Risk-sensitive Reinforcement Learningcited by 129
  3. 2020GenDICE: Generalized Offline Estimation of Stationary Valuescited by 122
  4. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  5. 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
  6. 2020Minimax-Optimal Off-Policy Evaluation with Linear Function Approximationcited by 120
  7. 2019Optimism in Reinforcement Learning with Generalized Linear Function Approximationcited by 111
  8. 2020The Importance of Pessimism in Fixed-Dataset Policy Optimizationcited by 99
  9. 2019Distributionally Robust Optimization: A Reviewcited by 75
  10. 2019Optimization for deep learning: theory and algorithmscited by 70
  11. 2020GradientDICE: Rethinking Generalized Offline Estimation of Stationary Valuescited by 70
  12. 2021The Modern Mathematics of Deep Learningcited by 63
  13. 2018Deep Neural Networks for Estimation and Inferencecited by 62
  14. 2021On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widthscited by 44
  15. 2019Distributionally Robust Reinforcement Learningcited by 40
  16. 2020Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimensioncited by 36
  17. 2018Bayesian Optimization with Expensive Integrandscited by 24
  18. 2021Combining Online Learning and Offline Learning for Contextual Bandits with Deficient Supportcited by 7
  19. 2019A Distributionally Robust Boosting Algorithmcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Understanding Reinforcement Learning Algorithms: The Progress from Basic Q-learning to Proximal Policy Optimization
  2. 2025Statistical and Algorithmic Foundations of Reinforcement Learning
  3. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review
  4. 2025On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures
  5. 2025Reinforcement Learning from Human Feedback
  6. 2024Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning
  7. 2026Robust General Utility for Reinforcement Learning
  8. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress