# A Reinforcement Learning Approach to Age of Information in Multi-User Networks with HARQ

arXiv: https://arxiv.org/abs/2102.09774 · HTML: https://mlsanity.com/paper/2102.09774/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 43 references that are in the map.
- First posted on arXiv 19 Feb 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Scheduling Policies for Minimizing Age of Information in Broadcast Wireless Networks](https://mlsanity.com/paper/1801.01803/) (2018)
- [Deep Reinforcement Learning for Minimizing Age-of-Information in UAV-assisted Networks](https://mlsanity.com/paper/1905.02993/) (2019)
- [Reinforcement Learning to Minimize Age of Information with an Energy Harvesting Sensor with HARQ and Sensing Cost](https://mlsanity.com/paper/1902.09467/) (2019)
- [Status Updates Through Multicast Networks](https://mlsanity.com/paper/1709.02427/) (2017)
- [A Reinforcement Learning Approach to Age of Information in Multi-User Networks](https://mlsanity.com/paper/1806.00336/) (2018)
- [Reinforcement Learning Based Scheduling Algorithm for Optimizing Age of Information in Ultra Reliable Low Latency Networks](https://mlsanity.com/paper/1811.06776/) (2018)
- [On Timely Channel Coding with Hybrid ARQ](https://mlsanity.com/paper/1905.03238/) (2019)

## Built on by (most cited first, 9 of 9)

- [Reinforcement Learning-powered Semantic Communication via Semantic Similarity](https://mlsanity.com/paper/2108.12121/) (2021)
- [Learning to Minimize Age of Information over an Unreliable Channel with Energy Harvesting](https://mlsanity.com/paper/2106.16037/) (2021)
- [Collaborative Optimization of the Age of Information under Partial Observability](https://mlsanity.com/paper/2312.12977/) (2023)
- [How to Minimize the Weighted Sum AoI in Multi-Source Status Update Systems: OMA or NOMA?](https://mlsanity.com/paper/2205.03143/) (2022)
- [Asynchronous Fractional Multi-Agent Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing](https://mlsanity.com/paper/2409.16832/) (2024)
- [Age Optimal Sampling for Unreliable Channels under Unknown Channel Statistics](https://mlsanity.com/paper/2412.18119/) (2024)
- [Age-of-information minimization under energy harvesting and non-stationary environment](https://mlsanity.com/paper/2504.04916/) (2025)
- [Policy Gradient Algorithms for Age-of-Information Cost Minimization](https://mlsanity.com/paper/2512.11990/) (2025)
- [Real-Time Hard Peak Age-of-Information Safety with No-Regret Learning](https://mlsanity.com/paper/2607.27626/) (2026)

## Abstract (arXiv)

Scheduling the transmission of time-sensitive information from a source node to multiple users over error-prone communication channels is studied with the goal of minimizing the long-term average age of information (AoI) at the users. A long-term average resource constraint is imposed on the source, which limits the average number of transmissions. The source can transmit only to a single user at each time slot, and after each transmission, it receives an instantaneous ACK/NACK feedback from the intended receiver, and decides when and to which user to transmit the next update. Assuming the channel statistics are known, the optimal scheduling policy is studied for both the standard automatic repeat request (ARQ) and hybrid ARQ (HARQ) protocols. Then, a reinforcement learning(RL) approach is introduced to find a near-optimal policy, which does not assume any a priori information on the random processes governing the channel states. Different RL methods including average-cost SARSAwith linear function approximation (LFA), upper confidence reinforcement learning (UCRL2), and deep Q-network (DQN) are applied and compared through numerical simulations