mlsanityData to 7 Oct 2026

NetworkGym: Reinforcement Learning Environments for Multi-Access Traffic Management in Network Simulation

arXiv 2411.04138 ↗First posted 30 Oct 2024cs.NI cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Mobile devices such as smartphones, laptops, and tablets can often connect to multiple access networks (e.g., Wi-Fi, LTE, and 5G) simultaneously. Recent advancements facilitate seamless integration of these connections below the transport layer, enhancing the experience for apps that lack inherent multi-path support. This optimization hinges on dynamically determining the traffic distribution across networks for each device, a process referred to as textitmulti-access traffic splitting. This paper introduces textitNetworkGym, a high-fidelity network environment simulator that facilitates generating multiple network traffic flows and multi-access traffic splitting. This simulator facilitates training and evaluating different RL-based solutions for the multi-access traffic splitting problem. Our initial explorations demonstrate that the majority of existing state-of-the-art offline RL algorithms (e.g. CQL) fail to outperform certain hand-crafted heuristic policies on average. This illustrates the urgent need to evaluate offline RL algorithms against a broader range of benchmarks, rather than relying solely on popular ones such as D4RL. We also propose an extension to the TD3+BC algorithm, named Pessimistic TD3 (PTD3), and demonstrate that it outperforms many state-of-the-art offline RL algorithms. PTD3's behavioral constraint mechanism, which relies on value-function pessimism, is theoretically motivated and relatively simple to implement.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 32 of its 59 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

32 of 59 references.

References · 32

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  9. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  10. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
  11. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  12. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
20 more
  1. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  2. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  3. 2021Offline RL Without Off-Policy Evaluationcited by 150
  4. 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
  5. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 117
  6. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
  7. 2021Towards Instance-Optimal Offline Reinforcement Learning with Pessimismcited by 78
  8. 2022Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimismcited by 63
  9. 2022On Instance-Dependent Bounds for Offline Reinforcement Learning with Linear Function Approximationcited by 22
  10. 2022Q-Ensemble for Offline RL: Don't Scale the Ensemble, Scale the Batch Sizecited by 20
  11. 2017Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-time Popularitiescited by 15
  12. 2019Neural Packet Classificationcited by 14
  13. 2022Automatic Curriculum Generation for Learning Adaptation in Networkingcited by 14
  14. 2022Offline Reinforcement Learning with Closed-Form Policy Improvement Operatorscited by 14
  15. 2022Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficientcited by 12
  16. 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasetscited by 12
  17. 2023Improving and Benchmarking Offline Reinforcement Learning Algorithmscited by 6
  18. 2022Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligencecited by 2
  19. 2022A Reinforcement Learning Approach to Optimize Available Network Bandwidth Utilizationcited by 2
  20. 2022Multi-Access Point Coordination for Next-Gen Wi-Fi Networks Aided by Deep Reinforcement Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025MuFlex: A Scalable, Physics-based Platform for Multi-Building Flexibility Analysis and Coordinationcited by 0
  2. 2026Towards Near-Real-Time Telemetry-Aware Routing with Neural Routing Algorithmscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Advancing RAN Slicing with Offline Reinforcement Learning
  2. 2024Large Vision Model-Enhanced Digital Twin with Deep Reinforcement Learning for User Association and Load Balancing in Dynamic Wireless Networks
  3. 2026Telecom World Models: Unifying Digital Twins, Foundation Models, and Predictive Planning for 6G
  4. 2025Generative Resource Allocation for 6G O-RAN with Diffusion Policies
  5. 2026Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
  6. 2025Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models
  7. 2026Large Language Model (LLM)-enabled Reinforcement Learning for Wireless Network Optimization
  8. 2026Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization