mlsanityData to 7 Oct 2026

Personalization for Web-based Services using Offline Reinforcement Learning

arXiv 2102.05612 ↗First posted 10 Feb 2021cs.LG cs.HC cs.SEcited by 3

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Large-scale Web-based services present opportunities for improving UI policies based on observed user interactions. We address challenges of learning such policies through model-free offline Reinforcement Learning (RL) with off-policy training. Deployed in a production system for user authentication in a major social network, it significantly improves long-term objectives. We articulate practical challenges, compare several ML techniques, provide insights on training and evaluation of RL models, and discuss generalizations.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 29 of its 74 references that are in the map. First posted on arXiv 10 Feb 2021.

Builds on

29 of 74 references.

Closest ancestors · 6

Same direction, newest first

  1. 2020Hyperparameter Selection for Offline Reinforcement Learning17 Jul
  2. 2020Provably Good Batch Reinforcement Learning Without Great Exploration16 Jul
  3. 2020Critic Regularized Regression26 Jun
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  5. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning19 Feb
  6. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  11. 2018Learning Dexterous In-Hand Manipulationcited by 643
  12. 2017Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learningcited by 475
11 more
  1. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  2. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 337
  3. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  4. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  5. 2019Challenges of Real-World Reinforcement Learningcited by 282
  6. 2017Neural Attentive Session-based Recommendationcited by 237
  7. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  8. 2021Reinforcement learning based recommender systems: A surveycited by 187
  9. 2018Horizon: Facebook's Open Source Applied Reinforcement Learning Platformcited by 48
  10. 2013Personalized News Recommendation with Context Treescited by 11
  11. 2019Predictive Precompute with Recurrent Neural Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 1.

3 papers.

  1. 2022Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Sizecited by 20
  2. 2021Looper: An end-to-end ML platform for product decisionscited by 9
  3. 2023Practical Knowledge Distillation: Using DNNs to Beat DNNscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Offline Reinforcement Learning for Human-Guided Human-Machine Interaction with Private Information
  2. 2021Offline Preference-Based Apprenticeship Learning
  3. 2024Preserving Expert-Level Privacy in Offline Reinforcement Learning
  4. 2023Benchmarks and Algorithms for Offline Preference-Based Reward Learning
  5. 2026Action-Free Offline-to-Online RL via Discretised State Policies
  6. 2022User-Interactive Offline Reinforcement Learning
  7. 2023Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Bias
  8. 2021A Minimalist Approach to Offline Reinforcement Learning