mlsanityData to 7 Oct 2026

Pearl: A Production-ready Reinforcement Learning Agent

arXiv 2312.03814 ↗First posted 6 Dec 2023cs.LG cs.AIcited by 9

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) is a versatile framework for optimizing long-term goals. Although many real-world problems can be formalized with RL, learning and deploying a performant RL policy requires a system designed to address several important challenges, including the exploration-exploitation dilemma, partial observability, dynamic action spaces, and safety concerns. While the importance of these challenges has been well recognized, existing open-source RL libraries do not explicitly address them. This paper introduces Pearl, a Production-Ready RL software package designed to embrace these challenges in a modular way. In addition to presenting benchmarking results, we also highlight examples of Pearl's ongoing industry adoption to demonstrate its advantages for production use cases. Pearl is open sourced on GitHub at github.com/facebookresearch/pearl and its official website is pearlagent.github.io.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 44 of its 84 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

44 of 84 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  8. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  11. 2023Mastering Diverse Domains through World Modelscited by 1,193
  12. 2017Hindsight Experience Replaycited by 1,096
32 more
  1. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  2. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  3. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 616
  5. 2016Deep Exploration via Bootstrapped DQNcited by 518
  6. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 494
  7. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  8. 2018Reward Constrained Policy Optimizationcited by 397
  9. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  10. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  11. 2019Challenges of Real-World Reinforcement Learningcited by 282
  12. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 230
  13. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  14. 2020Beyond UCB: Optimal and Efficient Contextual Bandits with Regression Oraclescited by 195
  15. 2017Deep Exploration via Randomized Value Functionscited by 155
  16. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  17. 2018Practical Contextual Bandits with Regression Oraclescited by 92
  18. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  19. 2020Neural Contextual Bandits with Deep Representation and Shallow Explorationcited by 71
  20. 2017Boltzmann Exploration Done Rightcited by 69
  21. 2018A Contextual Bandit Bake-offcited by 64
  22. 2017Ensemble Samplingcited by 62
  23. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  24. 2018Horizon: Facebook's Open Source Applied Reinforcement Learning Platformcited by 48
  25. 2021MBRL-Lib: A Modular Library for Model-based Reinforcement Learningcited by 41
  26. 2023Approximate Thompson Sampling via Epistemic Neural Networkscited by 31
  27. 2021MALib: A Parallel Framework for Population-based Multi-agent Reinforcement Learningcited by 23
  28. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  29. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 14
  30. 2022Recurrent networks, hidden states and beliefs in partially observable environmentscited by 11
  31. 2023Offline Reinforcement Learning for Optimizing Production Bidding Policiescited by 8
  32. 2023Optimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 3; 2026: 1.

9 papers.

  1. 2024AI Metropolis: Scaling Large Language Model-based Multi-Agent Simulation with Out-of-order Executioncited by 14
  2. 2024Advancing Investment Frontiers: Industry-grade Deep Reinforcement Learning for Portfolio Optimizationcited by 4
  3. 2024Simulation-Based Benchmarking of Reinforcement Learning Agents for Personalized Retail Promotionscited by 4
  4. 2026Creator Incentives in Recommender Systems: A Cooperative Game-Theoretic Approach for Stable and Fair Collaboration in Multi-Agent Banditscited by 2
  5. 2025An Empirical Study of Deep Reinforcement Learning in Continuing Taskscited by 1
  6. 2024The New Era of Dynamic Pricing: Synergizing Supervised Learning and Quadratic Programmingcited by 0
  7. 2024A Research Agenda for Usability and Generalisation in Reinforcement Learningcited by 0
  8. 2025ObjectRL: An Object-Oriented Reinforcement Learning Codebasecited by 0
  9. 2025Taxonomy and Trends in Reinforcement Learning for Robotics and Control Systems: A Structured Reviewcited by 0