mlsanityData to 7 Oct 2026

A Reference Architecture of Reinforcement Learning Frameworks

arXiv 2603.06413 ↗First posted 6 Mar 2026cs.SE cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The surge in reinforcement learning (RL) applications gave rise to diverse supporting technology, such as RL frameworks. However, the architectural patterns of these frameworks are inconsistent across implementations and there exists no reference architecture (RA) to form a common basis of comparison, evaluation, and integration. To address this gap, we propose an RA of RL frameworks. Through a grounded theory approach, we analyze 18 state-of-the-practice RL frameworks and, by that, we identify recurring architectural components and their relationships, and codify them in an RA. To demonstrate our RA, we reconstruct characteristic RL patterns. Finally, we identify architectural trends, e.g., commonly used components, and outline paths to improving RL frameworks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 82 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

39 of 82 references.

References · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  7. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  8. 2018Learning Dexterous In-Hand Manipulationcited by 643
  9. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  10. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  11. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  12. 2020dm_control: Software and Tasks for Continuous Controlcited by 368
27 more
  1. 2018Learning to Drive in a Daycited by 266
  2. 2024Reinforcement learningcited by 236
  3. 2018Unity: A General Platform for Intelligent Agentscited by 232
  4. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  5. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  6. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 189
  7. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  8. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 139
  9. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  10. 2016DeepMind Labcited by 74
  11. 2023Jumanji: a Diverse Suite of Scalable Reinforcement Learning Environments in JAXcited by 52
  12. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 36
  13. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  14. 2021Reverb: A Framework For Experience Replaycited by 27
  15. 2023A Meta-Summary of Challenges in Building Products with ML Components -- Collecting Experiences from 4758+ Practitionerscited by 22
  16. 2022MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Librarycited by 21
  17. 2021Launchpad: A Programming Model for Distributed Machine Learning Researchcited by 14
  18. 2021Mava: a research library for distributed multi-agent reinforcement learning in JAXcited by 14
  19. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Surveycited by 11
  20. 2020Selecting third-party libraries: The practitioners' perspectivecited by 9
  21. 2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
  22. 2019Conditional Vehicle Trajectories Prediction in CARLA Urban Environmentcited by 6
  23. 2025AI Simulation by Digital Twins: Systematic Survey, Reference Framework, and Mapping to a Standardized Architecturecited by 6
  24. 2023Common Challenges of Deep Reinforcement Learning Applications Development: An Empirical Studycited by 5
  25. 2022A Comparison of Reinforcement Learning Frameworks for Software Testing Taskscited by 3
  26. 2025Complex Model Transformations by Reinforcement Learning with Uncertain Human Guidancecited by 2
  27. 2021Sim-Env: Decoupling OpenAI Gym Environments from Simulation Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Trust the AI, Doubt Yourself: The Effect of Urgency on Self-Confidence in Human-AI Interactioncited by 0
  2. 2026A Model-Driven Approach for Developing Families of Reinforcement Learning Environmentscited by 0
  3. 2026Understanding the Architecture of Coding Agents: An Exploratory Study Using a Research Prototypecited by 0