# Model Based Residual Policy Learning with Applications to Antenna Control

arXiv: https://arxiv.org/abs/2211.08796 · HTML: https://mlsanity.com/paper/2211.08796/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 28 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [DeepMind Control Suite](https://mlsanity.com/paper/1801.00690/) (2018)
- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Residual Policy Learning](https://mlsanity.com/paper/1812.06298/) (2018)
- [MBRL-Lib: A Modular Library for Model-based Reinforcement Learning](https://mlsanity.com/paper/2104.10159/) (2021)
- [Optimizing Coverage and Capacity in Cellular Networks using Machine Learning](https://mlsanity.com/paper/2010.13710/) (2020)
- [Coordinated Reinforcement Learning for Optimizing Mobile Networks](https://mlsanity.com/paper/2109.15175/) (2021)
- [Remote Electrical Tilt Optimization via Safe Reinforcement Learning](https://mlsanity.com/paper/2010.05842/) (2020)
- [Symbolic Reinforcement Learning for Safe RAN Control](https://mlsanity.com/paper/2103.06602/) (2021)
- [A Safe Reinforcement Learning Architecture for Antenna Tilt Optimisation](https://mlsanity.com/paper/2012.01296/) (2020)

## Built on by (most cited first, 1 of 1)

- [Offline Reinforcement Learning and Sequence Modeling for Downlink Link Adaptation](https://mlsanity.com/paper/2410.23031/) (2024)

## Abstract (arXiv)

Non-differentiable controllers and rule-based policies are widely used for controlling real systems such as telecommunication networks and robots. Specifically, parameters of mobile network base station antennas can be dynamically configured by these policies to improve users coverage and quality of service. Motivated by the antenna tilt control problem, we introduce Model-Based Residual Policy Learning (MBRPL), a practical reinforcement learning (RL) method. MBRPL enhances existing policies through a model-based approach, leading to improved sample efficiency and a decreased number of interactions with the actual environment when compared to off-the-shelf RL methods.To the best of our knowledge, this is the first paper that examines a model-based approach for antenna control. Experimental results reveal that our method delivers strong initial performance while improving sample efficiency over previous RL methods, which is one step towards deploying these algorithms in real networks.