mlsanityData to 14 Oct 2026

Robust Bandwidth Estimation for Real-Time Communication with Offline Reinforcement Learning

arXiv 2507.05785 ↗First posted 8 Jul 2025eess.SY cs.LG cs.SYcited by 0

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

Accurate bandwidth estimation (BWE) is critical for real-time communication (RTC) systems. Traditional heuristic approaches offer limited adaptability under dynamic networks, while online reinforcement learning (RL) suffers from high exploration costs and potential service disruptions. Offline RL, which leverages high-quality data collected from real-world environments, offers a promising alternative. However, challenges such as out-of-distribution (OOD) actions, policy extraction from behaviorally diverse datasets, and reliable deployment in production systems remain unsolved. We propose RBWE, a robust bandwidth estimation framework based on offline RL that integrates Q-ensemble (an ensemble of Q-functions) with a Gaussian mixture policy to mitigate OOD risks and enhance policy learning. A fallback mechanism ensures deployment stability by switching to heuristic methods under high uncertainty. Experimental results show that RBWE reduces overestimation errors by 18% and improves the 10th percentile Quality of Experience (QoE) by 18.6%, demonstrating its practical effectiveness in real-world RTC applications. The implementation is publicly available at https://github.com/jiu2021/RBWE_offline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 17 references that are in the map. First posted on arXiv 8 Jul 2025.

Builds on

6 of 17 references.

References · 6

  1. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  2. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
  3. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 231
  4. 2024ACM MMSys 2024 Bandwidth Estimation in Real Time Communications Challengecited by 2
  5. 2024Mowgli: Passively Learned Rate Control for Real-Time Videocited by 2
  6. 2023From Ember to Blaze: Swift Interactive Video Adaptation via Meta-Reinforcement Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTC
  2. 2025Human-in-the-Loop Bandwidth Estimation for Quality of Experience Optimization in Real-Time Video Communication
  3. 2025Offline and Distributional Reinforcement Learning for Wireless Communications
  4. 2023Towards Robust Offline-to-Online Reinforcement Learning via Uncertainty and Smoothness
  5. 2024Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation
  6. 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets
  7. 2026Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
  8. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruption