mlsanityData to 14 Oct 2026

A Multi-Agent Multi-Environment Mixed Q-Learning for Partially Decentralized Wireless Network Optimization

arXiv 2409.16450 ↗First posted 24 Sep 2024eess.SP cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Q-learning is a powerful tool for network control and policy optimization in wireless networks, but it struggles with large state spaces. Recent advancements, like multi-environment mixed Q-learning (MEMQ), improves performance and reduces complexity by integrating multiple Q-learning algorithms across multiple related environments so-called digital cousins. However, MEMQ is designed for centralized single-agent networks and is not suitable for decentralized or multi-agent networks. To address this challenge, we propose a novel multi-agent MEMQ algorithm for partially decentralized wireless networks with multiple mobile transmitters (TXs) and base stations (BSs), where TXs do not have access to each other's states and actions. In uncoordinated states, TXs act independently to minimize their individual costs. In coordinated states, TXs use a Bayesian approach to estimate the joint state based on local observations and share limited information with leader TX to minimize joint cost. The cost of information sharing scales linearly with the number of TXs and is independent of the joint state-action space size. The proposed scheme is 50% faster than centralized MEMQ with only a 20% increase in average policy error (APE) and is 25% faster than several advanced decentralized Q-learning algorithms with 40% less APE. The convergence of the algorithm is also demonstrated.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 30 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

6 of 30 references.

References · 6

  1. 2020Maxmin Q-learning: Controlling the Estimation Bias of Q-learningcited by 117
  2. 2016Averaged-DQN: Variance Reduction and Stabilization for Deep Reinforcement Learningcited by 106
  3. 2020A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraintcited by 7
  4. 2024Multi-Timescale Ensemble Q-learning for Markov Decision Process Policy Optimizationcited by 5
  5. 2024Leveraging Digital Cousins for Ensemble Q-Learning in Large-Scale Wireless Networkscited by 4
  6. 2024Coverage Analysis of Multi-Environment Q-Learning Algorithms for Wireless Network Optimizationcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Coverage Analysis for Digital Cousin Selection -- Improving Multi-Environment Q-Learningcited by 1
  2. 2025Partially Decentralized Multi-Agent Q-Learning via Digital Cousins for Wireless Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Foundation Model-Aided Multi-Agent Reinforcement Learning for Wireless Random Access Network Optimization
  2. 2024Multi-Agent Q-Learning for Real-Time Load Balancing User Association and Handover in Mobile Networks
  3. 2025An Offline Multi-Agent Reinforcement Learning Framework for Radio Resource Management
  4. 2025QoS-aware State-Augmented Learnable Algorithm for Wireless Coexistence Parameter Management
  5. 2024Heterogeneous Multi-Agent Reinforcement Learning for Distributed Channel Access in WLANs
  6. 2026Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks
  7. 2026Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints
  8. 2025Consensus-based Decentralized Multi-agent Reinforcement Learning for Random Access Network Optimization