mlsanityData to 14 Oct 2026

RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN

arXiv 2111.06978 ↗First posted 12 Nov 2021cs.NI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

Radio access network (RAN) technologies continue to evolve, with Open RAN gaining the most recent momentum. In the O-RAN specifications, the RAN intelligent controllers (RICs) are software-defined orchestration and automation functions for the intelligent management of RAN. This article introduces principles for machine learning (ML), in particular, reinforcement learning (RL) applications in the O-RAN stack. Furthermore, we review the state-of-the-art research in wireless networks and cast it onto the RAN framework and the hierarchy of the O-RAN architecture. We provide a taxonomy for the challenges faced by ML/RL models throughout the development life-cycle: from the system specification to production deployment (data acquisition, model design, testing and management, etc.). To address the challenges, we integrate a set of existing MLOps principles with unique characteristics when RL agents are considered. This paper discusses a systematic model development, testing and validation life-cycle, termed: RLOps. We discuss fundamental parts of RLOps, which include: model specification, development, production environment serving, operations monitoring and safety/security. Based on these principles, we propose the best practices for RLOps to achieve an automated and reproducible model development process. At last, a holistic data analytics platform rooted in the O-RAN deployment is designed and implemented, aiming to embrace and fulfil the aforementioned principles and best practices of RLOps.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 39 of its 97 references that are in the map. First posted on arXiv 12 Nov 2021.

Builds on

39 of 97 references.

References · 39

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2016OpenAI Gymcited by 1,736
  4. 2016Concrete Problems in AI Safetycited by 1,642
  5. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,028
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  7. 2017Deep Reinforcement Learning that Matterscited by 754
  8. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  9. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  10. 2020Learning under Concept Drift: A Reviewcited by 477
  11. 2018Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Surveycited by 460
  12. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
27 more
  1. 2017Deep Reinforcement Learning: An Overviewcited by 333
  2. 2018Distributed Prioritized Experience Replaycited by 269
  3. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 154
  4. 2019AliGraph: A Comprehensive Graph Neural Network Platformcited by 153
  5. 2020Explainable Reinforcement Learning: A Surveycited by 141
  6. 2017Stochastic Neural Networks for Hierarchical Reinforcement Learningcited by 124
  7. 2018A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspectivecited by 112
  8. 2019Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learningcited by 93
  9. 2018Deep Learning-Based Channel Estimationcited by 75
  10. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  11. 2018Horizon: Facebook's Open Source Applied Reinforcement Learning Platformcited by 48
  12. 2020Intelligence and Learning in O-RAN for Data-driven NextG Cellular Networkscited by 45
  13. 2020Evaluating the Performance of Reinforcement Learning Algorithmscited by 36
  14. 2018Learning Curriculum Policies for Reinforcement Learningcited by 35
  15. 2020Learning with Safety Constraints: Sample Complexity of Reinforcement Learning for Constrained MDPscited by 32
  16. 2021A Data Quality-Driven View of MLOpscited by 31
  17. 2018OFDM-Autoencoder for End-to-End Learning of Communications Systemscited by 29
  18. 2018Optimized Computation Offloading Performance in Virtual Edge Computing Systems via Deep Reinforcement Learningcited by 26
  19. 2020Robust Constrained-MDPs: Soft-Constrained Robust Policy Optimization under Model Uncertaintycited by 23
  20. 2021Measuring Sample Efficiency and Generalization in Reinforcement Learning Benchmarks: NeurIPS 2020 Procgen Benchmarkcited by 23
  21. 2020Open, Programmable, and Virtualized 5G Networks: State-of-the-Art and the Road Aheadcited by 15
  22. 2019Deep Learning based Downlink Channel Prediction for FDD Massive MIMO Systemcited by 13
  23. 2020How to Make Deep RL Work in Practicecited by 8
  24. 2021Self-play Learning Strategies for Resource Assignment in Open-RAN Networkscited by 8
  25. 2020Reinforcement Learning Based Dynamic Function Splitting in Disaggregated Green Open RANscited by 7
  26. 2021The Next Decade of Telecommunications Artificial Intelligencecited by 3
  27. 2017Graph of Virtual Actors (GOVA): a Big Data Analytics Architecture for IoTcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 5; 2023: 4; 2024: 3; 2025: 1; 2026: 2.

15 papers.

Sort:

By area:

  1. 2023Safe and Accelerated Deep Reinforcement Learning-based O-RAN Slicing: A Hybrid Transfer Learning Approachcited by 5
  2. 2026ML-Enabled Open RAN: A Comprehensive Survey of Architectures, Challenges, and Opportunitiescited by 4
  3. 2022Sim2real for Reinforcement Learning Driven Next Generation Networkscited by 3
  4. 2022Transmit Power Control for Indoor Small Cells: A Method Based on Federated Reinforcement Learningcited by 3
  5. 2022Intelligent Closed-loop RAN Control with xApps in OpenRAN Gymcited by 2
  6. 2024Adapting MLOps for Diverse In-Network Intelligence in 6G Era: Challenges and Solutionscited by 2
  7. 2023An Intentional Forgetting-Driven Self-Healing Method For Deep Reinforcement Learning Systemscited by 1
  8. 2024Past, Present, Future: A Comprehensive Exploration of AI Use Cases in the UMBRELLA IoT Testbedcited by 1
  9. 2024Building the Self-Improvement Loop: Error Detection and Correction in Goal-Oriented Semantic Communicationscited by 1
  10. 2022Variational Autoencoder Assisted Neural Network Likelihood RSRP Prediction Modelcited by 0
  11. 2022FORLORN: A Framework for Comparing Offline Methods and Reinforcement Learning for Optimization of RAN Parameterscited by 0
  12. 2023Sparsity-Aware Intelligent Massive Random Access Control in Open RAN: A Reinforcement Learning Based Approachcited by 0
3 more
  1. 2023Demo: A Digital Twin of the 5G Radio Access Network for Anomaly Detection Functionalitycited by 0
  2. 2025Federated Neuroevolution O-RAN: Enhancing the Robustness of Deep Reinforcement Learning xAppscited by 0
  3. 2026Latency-aware Human-in-the-Loop Reinforcement Learning for Semantic Communicationscited by 0