# Zero-Shot Adaptation for mmWave Beam-Tracking on Overhead Messenger Wires through Robust Adversarial Reinforcement Learning

arXiv: https://arxiv.org/abs/2102.08055 · HTML: https://mlsanity.com/paper/2102.08055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 53 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Millimeter Wave Cellular Wireless Networks: Potentials and Challenges](https://mlsanity.com/paper/1401.2560/) (2014)
- [Deep Reinforcement Learning for 5G Networks: Joint Beamforming, Power Control, and Interference Coordination](https://mlsanity.com/paper/1907.00123/) (2019)
- [Massive Autonomous UAV Path Planning: A Neural Network Based Mean-Field Game Theoretic Approach](https://mlsanity.com/paper/1905.04152/) (2019)
- [Joint Antenna Selection and Hybrid Beamformer Design using Unquantized and Quantized Deep Learning Networks](https://mlsanity.com/paper/1905.03107/) (2019)
- [Adaptive Beam Tracking with the Unscented Kalman Filter for Millimeter Wave Communication](https://mlsanity.com/paper/1804.08640/) (2018)
- [Millimeter Wave Communications on Overhead Messenger Wire: Deep Reinforcement Learning-Based Predictive Beam Tracking](https://mlsanity.com/paper/2012.00982/) (2020)
- [Statistical Approaches for Initial Access in mmWave 5G Systems](https://mlsanity.com/paper/1711.05456/) (2017)
- [Study of Realistic Antenna Patterns in 5G mmWave Cellular Scenarios](https://mlsanity.com/paper/1802.01316/) (2018)
- [Fast Channel Estimation and Beam Tracking for Millimeter Wave Vehicular Communications](https://mlsanity.com/paper/1806.00161/) (2018)

## Abstract (arXiv)

Millimeter wave (mmWave) beam-tracking based on machine learning enables the development of accurate tracking policies while obviating the need to periodically solve beam-optimization problems. However, its applicability is still arguable when training-test gaps exist in terms of environmental parameters that affect the node dynamics. From this skeptical point of view, the contribution of this study is twofold. First, by considering an example scenario, we confirm that the training-test gap adversely affects the beam-tracking performance. More specifically, we consider nodes placed on overhead messenger wires, where the node dynamics are affected by several environmental parameters, e.g, the wire mass and tension. Although these are particular scenarios, they yield insight into the validation of the training-test gap problems. Second, we demonstrate the feasibility of textitzero-shot adaptation as a solution, where a learning agent adapts to environmental parameters unseen during training. This is achieved by leveraging a robust adversarial reinforcement learning (RARL) technique, where such training-and-test gaps are regarded as disturbances by adversaries that are jointly trained with a legitimate beam-tracking agent. Numerical evaluations demonstrate that the beam-tracking policy learned via RARL can be applied to a wide range of environmental parameters without severely degrading the received power.