mlsanityData to 14 Oct 2026

Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control

arXiv 2603.03932 ↗First posted 4 Mar 2026cs.NI cs.AI cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

Offline Reinforcement Learning (RL) is a promising approach for next-generation wireless networks, where online exploration is unsafe and large amounts of operational data can be reused across the model lifecycle. However, the behavior of offline RL algorithms under genuinely stochastic dynamics -- inherent to wireless systems due to fading, noise, and traffic mobility -- remains insufficiently understood. We address this gap by evaluating Bellman-based (Conservative Q-Learning), sequence-based (Decision Transformers), and hybrid (Critic-Guided Decision Transformers) offline RL methods in an open-access stochastic telecom environment (mobile-env). Our results show that Conservative Q-Learning consistently produces more robust policies across different sources of stochasticity, making it a reliable default choice in lifecycle-driven AI management frameworks. Sequence-based methods remain competitive and can outperform Bellman-based approaches when sufficient high-return trajectories are available. These findings provide practical guidance for offline RL algorithm selection in AI-driven network control pipelines, such as O-RAN and future 6G functions, where robustness and data availability are key operational constraints.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 25 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

19 of 25 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Offline Reinforcement Learning for Mobility Robustness Optimization28 Jun
  2. 2024Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation30 Oct
  3. 2023Advancing RAN Slicing with Offline Reinforcement Learning16 Dec
  4. 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets19 Nov

Other references · 15

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  7. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  8. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  9. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 63
  10. 2022A Review of Uncertainty for Deep Reinforcement Learningcited by 40
  11. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learningcited by 26
  12. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 25
3 more
  1. 2023When should we prefer Decision Transformers for Offline Reinforcement Learning?cited by 20
  2. 2024Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modelingcited by 18
  3. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Offline and Distributional Reinforcement Learning for Wireless Communications
  2. 2026Offline Multi-Agent Reinforcement Learning for 6G Communications: Fundamentals, Applications and Future Directions
  3. 2026Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management
  4. 2024Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTC
  5. 2025A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory
  6. 2026Diffusion Offline Reinforcement Learning for Fair and Energy-Efficient UAV-Assisted Wireless Networks
  7. 2025A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms
  8. 2024Offline and Distributional Reinforcement Learning for Radio Resource Management