mlsanityData to 7 Oct 2026

Challenges of Real-World Reinforcement Learning

arXiv 1904.12901 ↗First posted 29 Apr 2019cs.LG cs.AI cs.ROcited by 282

Not on the map. Why?

Cited by 32 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are often hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. We present a set of nine unique challenges that must be addressed to productionize RL to real world problems. For each of these challenges, we specify the exact meaning of the challenge, present some approaches from the literature, and specify some metrics for evaluating that challenge. An approach that addresses all nine challenges would be applicable to a large number of real world problems. We also present an example domain that has been modified to present these challenges as a testbed for practical RL research.

Key facts. Cited by 282 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 6; 2021: 73; 2022: 50; 2023: 42; 2024: 35; 2025: 37; 2026: 38.

282 papers.

Sort:

By area:

  1. 2023Learning Interactive Real-World Simulatorscited by 440
  2. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  3. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  4. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  5. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  6. 2021The Benchmark Lotterycited by 133
  7. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  8. 2023On Generative Agents in Recommendationcited by 88
  9. 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 78
  10. 2021Risk-Averse Offline Reinforcement Learningcited by 65
  11. 2025WorldGym: World Model as An Environment for Policy Evaluationcited by 61
  12. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learningcited by 57
38 more
  1. 2021Fault-Tolerant Federated Reinforcement Learning with Theoretical Guaranteecited by 52
  2. 2021Breaking the Deadly Triad with a Target Networkcited by 47
  3. 2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
  4. 2021Universal Off-Policy Evaluationcited by 43
  5. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  6. 2021Zero Time Waste: Recycling Predictions in Early Exit Neural Networkscited by 39
  7. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 36
  8. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 35
  9. 2021ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Marketscited by 34
  10. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatchcited by 33
  11. 2022Stabilizing Off-Policy Deep Reinforcement Learning from Pixelscited by 32
  12. 2022Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capturecited by 31
  13. 2021Average-Reward Off-Policy Policy Evaluation with Function Approximationcited by 30
  14. 2021Direct Behavior Specification via Constrained Reinforcement Learningcited by 30
  15. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 29
  16. 2021Acting in Delayed Environments with Non-Stationary Markov Policiescited by 28
  17. 2021Offline Reinforcement Learning with Pseudometric Learningcited by 28
  18. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  19. 2022Human-level Atari 200x fastercited by 28
  20. 2023PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learningcited by 28
  21. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  22. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  23. 2021Continuous Doubly Constrained Batch Reinforcement Learningcited by 27
  24. 2021A Dataset Perspective on Offline Reinforcement Learningcited by 27
  25. 2021Active Offline Policy Selectioncited by 26
  26. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 26
  27. 2022Hyperbolic Deep Reinforcement Learningcited by 25
  28. 2022Controlling Commercial Cooling Systems Using Reinforcement Learningcited by 25
  29. 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
  30. 2021Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Studycited by 24
  31. 2021MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learningcited by 24
  32. 2022An Empirical Study of Implicit Regularization in Deep Offline RLcited by 24
  33. 2024One-Shot Safety Alignment for Large Language Models via Optimal Dualizationcited by 24
  34. 2021Multi-Objective SPIBB: Seldonian Offline Policy Improvement with Safety Constraints in Finite MDPscited by 22
  35. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 22
  36. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  37. 2021RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystemscited by 21
  38. 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21

50 most cited of 282 All 282 (JSON)