mlsanityData to 14 Oct 2026

Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation

arXiv 2410.23031 ↗First posted 30 Oct 2024cs.LG cs.AI cs.SYcited by 3

On the map: Offline RL and imitation learning from data → Offline multi-agent RL

Abstract (arXiv)

Link adaptation (LA) is an essential function in modern wireless communication systems that dynamically adjusts the transmission rate of a communication link to match time- and frequency-varying radio link conditions. However, factors such as user mobility, fast fading, imperfect channel quality information, and aging of measurements make the modeling of LA challenging. To bypass the need for explicit modeling, recent research has introduced online reinforcement learning (RL) approaches as an alternative to the more commonly used rule-based algorithms. Yet, RL-based approaches face deployment challenges, as training in live networks can potentially degrade real-time performance. To address this challenge, this paper considers offline RL as a candidate to learn LA policies with minimal effects on the network operation. We propose three LA designs based on batch-constrained deep Q-learning, conservative Q-learning, and decision transformer. Our results show that offline RL algorithms can match the performance of state-of-the-art online RL methods when data is collected with a proper behavioral policy.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 35 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

18 of 35 references.

References · 18

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  4. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  6. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  7. 2020Transformer Hawkes Processcited by 235
  8. 2015Contextual Markov Decision Processescited by 220
  9. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 185
  10. 2022Mildly Conservative Q-Learning for Offline Reinforcement Learningcited by 112
  11. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 111
  12. 2022Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizescited by 73
6 more
  1. 2021A Dataset Perspective on Offline Reinforcement Learningcited by 27
  2. 2020Reinforcement Learning for Efficient and Tuning-Free Link Adaptationcited by 7
  3. 2024Model-Free Active Exploration in Reinforcement Learningcited by 5
  4. 2023Design Principles for Model Generalization and Scalable AI Integration in Radio Access Networkscited by 3
  5. 2022Model Based Residual Policy Learning with Applications to Antenna Controlcited by 1
  6. 2023Constrained Deep Reinforcement Learning for Fronthaul Compression Optimizationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Offline Reinforcement Learning for Mobility Robustness Optimizationcited by 1
  2. 2026Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Controlcited by 0
  3. 2026Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Offline Reinforcement Learning for Wireless Network Optimization with Mixture Datasets
  2. 2025Offline and Distributional Reinforcement Learning for Wireless Communications
  3. 2025Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL
  4. 2024Offline and Distributional Reinforcement Learning for Radio Resource Management
  5. 2025Pretraining a Shared Q-Network for Data-Efficient Offline Reinforcement Learning
  6. 2026Offline Multi-Agent Reinforcement Learning for 6G Communications: Fundamentals, Applications and Future Directions
  7. 2022MOORe: Model-based Offline-to-Online Reinforcement Learning
  8. 2026Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management