# Online Antenna Tuning in Heterogeneous Cellular Networks with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1903.06787 · HTML: https://mlsanity.com/paper/1903.06787/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2019.

## Built on by (most cited first, 13 of 13)

- [Coordinated Reinforcement Learning for Optimizing Mobile Networks](https://mlsanity.com/paper/2109.15175/) (2021)
- [Cellular Network Capacity and Coverage Enhancement with MDT Data and Deep Reinforcement Learning](https://mlsanity.com/paper/2202.10968/) (2022)
- [Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning](https://mlsanity.com/paper/2302.01199/) (2023)
- [A Graph Attention Learning Approach to Antenna Tilt Optimization](https://mlsanity.com/paper/2112.14843/) (2021)
- [Coverage and Capacity Optimization in STAR-RISs Assisted Networks: A Machine Learning Approach](https://mlsanity.com/paper/2204.06390/) (2022)
- [DRL Enabled Coverage and Capacity Optimization in STAR-RIS Assisted Networks](https://mlsanity.com/paper/2209.00511/) (2022)
- [Multi-Agent Reinforcement Learning with Common Policy for Antenna Tilt Optimization](https://mlsanity.com/paper/2302.12899/) (2023)
- [Continual Model-based Reinforcement Learning for Data Efficient Wireless Network Optimisation](https://mlsanity.com/paper/2404.19462/) (2024)
- [Safe RAN control: A Symbolic Reinforcement Learning Approach](https://mlsanity.com/paper/2106.01977/) (2021)
- [Learning Optimal Antenna Tilt Control Policies: A Contextual Linear Bandit Approach](https://mlsanity.com/paper/2201.02169/) (2022)
- [Learning Cellular Coverage from Real Network Configurations using GNNs](https://mlsanity.com/paper/2304.10328/) (2023)
- [Cooperative Tri-Point Model-Based Ground-to-Air Coverage Extension in Beyond 5G Networks](https://mlsanity.com/paper/2401.09757/) (2024)
- [Optimizing Reinforcement Learning Training over Digital Twin Enabled Multi-fidelity Networks](https://mlsanity.com/paper/2603.08931/) (2026)

## Abstract (arXiv)

We aim to jointly optimize antenna tilt angle, and vertical and horizontal half-power beamwidths of the macrocells in a heterogeneous cellular network (HetNet). The interactions between the cells, most notably due to their coupled interference render this optimization prohibitively complex. Utilizing a single agent reinforcement learning (RL) algorithm for this optimization becomes quite suboptimum despite its scalability, whereas multi-agent RL algorithms yield better solutions at the expense of scalability. Hence, we propose a compromise algorithm between these two. Specifically, a multi-agent mean field RL algorithm is first utilized in the offline phase so as to transfer information as features for the second (online) phase single agent RL algorithm, which employs a deep neural network to learn users locations. This two-step approach is a practical solution for real deployments, which should automatically adapt to environmental changes in the network. Our results illustrate that the proposed algorithm approaches the performance of the multi-agent RL, which requires millions of trials, with hundreds of online trials, assuming relatively low environmental dynamics, and performs much better than a single agent RL. Furthermore, the proposed algorithm is compact and implementable, and empirically appears to provide a performance guarantee regardless of the amount of environmental dynamics.