mlsanityData to 14 Oct 2026

Streetwise Agents: Empowering Offline RL Policies to Outsmart Exogenous Stochastic Disturbances in RTC

arXiv 2411.06815 ↗First posted 11 Nov 2024cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

The difficulty of exploring and training online on real production systems limits the scope of real-time online data/feedback-driven decision making. The most feasible approach is to adopt offline reinforcement learning from limited trajectory samples. However, after deployment, such policies fail due to exogenous factors that temporarily or permanently disturb/alter the transition distribution of the assumed decision process structure induced by offline samples. This results in critical policy failures and generalization errors in sensitive domains like Real-Time Communication (RTC). We solve this crucial problem of identifying robust actions in presence of domain shifts due to unseen exogenous stochastic factors in the wild. As it is impossible to learn generalized offline policies within the support of offline data that are robust to these unseen exogenous disturbances, we propose a novel post-deployment shaping of policies (Streetwise), conditioned on real-time characterization of out-of-distribution sub-spaces. This leads to robust actions in bandwidth estimation (BWE) of network bottlenecks in RTC and in standard benchmarks. Our extensive experimental results on BWE and other standard offline RL benchmark environments demonstrate a significant improvement (≈ 18% on some scenarios) in final returns wrt. end-user metrics over state-of-the-art baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 58 references that are in the map. First posted on arXiv 11 Nov 2024.

Builds on

11 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Offline to Online Learning for Real-Time Bandwidth Estimation23 Sep

Other references · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 562
  4. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 215
  5. 2024Is Value Learning Really the Main Bottleneck in Offline RL?cited by 89
  6. 2022Offline RL Policies Should be Trained to be Adaptivecited by 52
  7. 2022Masked Autoencoding for Scalable and Generalizable Decision Makingcited by 45
  8. 2019Reinforcement learning for bandwidth estimation and congestion control in real-time communicationscited by 6
  9. 2024Offline Reinforcement Learning from Datasets with Structured Non-Stationaritycited by 5
  10. 2024Offline Meta-learning for Real-time Bandwidth Estimationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Robust Bandwidth Estimation for Real-Time Communication with Offline Reinforcement Learning
  2. 2026Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control
  3. 2025Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization
  4. 2025Offline vs. Online Learning in Model-based RL: Lessons for Data Collection Strategies
  5. 2023Offline Data Enhanced On-Policy Policy Gradient with Provable Guarantees
  6. 2025A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory
  7. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation
  8. 2026Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management