# Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation

arXiv: https://arxiv.org/abs/2410.23031 · HTML: https://mlsanity.com/paper/2410.23031/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Offline multi-agent RL, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 30 Oct 2024.

## Placed in

- Offline RL and imitation learning from data / [Offline multi-agent RL](https://mlsanity.com/offlinerl/offline-multi-agent-rl/)

## Builds on (most cited first, 18 of 18)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [RvS: What is Essential for Offline RL via Supervised Learning?](https://mlsanity.com/paper/2112.10751/) (2021)
- [Mildly Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2206.04745/) (2022)
- [Benchmarking Batch Deep Reinforcement Learning Algorithms](https://mlsanity.com/paper/1910.01708/) (2019)
- [Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes](https://mlsanity.com/paper/2211.15144/) (2022)
- [A Dataset Perspective on Offline Reinforcement Learning](https://mlsanity.com/paper/2111.04714/) (2021)
- [Reinforcement Learning for Efficient and Tuning-Free Link Adaptation](https://mlsanity.com/paper/2010.08651/) (2020)
- [Model-Free Active Exploration in Reinforcement Learning](https://mlsanity.com/paper/2407.00801/) (2024)
- [Design Principles for Model Generalization and Scalable AI Integration in Radio Access Networks](https://mlsanity.com/paper/2306.06251/) (2023)
- [Model Based Residual Policy Learning with Applications to Antenna Control](https://mlsanity.com/paper/2211.08796/) (2022)
- [Constrained Deep Reinforcement Learning for Fronthaul Compression Optimization](https://mlsanity.com/paper/2309.15060/) (2023)

## Built on by (most cited first, 3 of 3)

- [Offline Reinforcement Learning for Mobility Robustness Optimization](https://mlsanity.com/paper/2506.22793/) (2025)
- [Selecting Offline Reinforcement Learning Algorithms for Stochastic Network Control](https://mlsanity.com/paper/2603.03932/) (2026)
- [Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networks](https://mlsanity.com/paper/2606.05208/) (2026)

## Abstract (arXiv)

Link adaptation (LA) is an essential function in modern wireless communication systems that dynamically adjusts the transmission rate of a communication link to match time- and frequency-varying radio link conditions. However, factors such as user mobility, fast fading, imperfect channel quality information, and aging of measurements make the modeling of LA challenging. To bypass the need for explicit modeling, recent research has introduced online reinforcement learning (RL) approaches as an alternative to the more commonly used rule-based algorithms. Yet, RL-based approaches face deployment challenges, as training in live networks can potentially degrade real-time performance. To address this challenge, this paper considers offline RL as a candidate to learn LA policies with minimal effects on the network operation. We propose three LA designs based on batch-constrained deep Q-learning, conservative Q-learning, and decision transformer. Our results show that offline RL algorithms can match the performance of state-of-the-art online RL methods when data is collected with a proper behavioral policy.