# Deep Reinforcement Learning-Based Beam Tracking for Low-Latency Services in Vehicular Networks

arXiv: https://arxiv.org/abs/2002.05564 · HTML: https://mlsanity.com/paper/2002.05564/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2020.

## Built on by (most cited first, 4 of 4)

- [Is High Variance Unavoidable in RL? A Case Study in Continuous Control](https://mlsanity.com/paper/2110.11222/) (2021)
- [Deep Learning Assisted Calibrated Beam Training for Millimeter-Wave Communication Systems](https://mlsanity.com/paper/2101.05206/) (2021)
- [Low-Precision Reinforcement Learning: Running Soft Actor-Critic in Half Precision](https://mlsanity.com/paper/2102.13565/) (2021)
- [Deep Learning Beam Optimization in Millimeter-Wave Communication Systems](https://mlsanity.com/paper/2107.07846/) (2021)

## Abstract (arXiv)

Ultra-Reliable and Low-Latency Communications (URLLC) services in vehicular networks on millimeter-wave bands present a significant challenge, considering the necessity of constantly adjusting the beam directions. Conventional methods are mostly based on classical control theory, e.g., Kalman filter and its variations, which mainly deal with stationary scenarios. Therefore, severe application limitations exist, especially with complicated, dynamic Vehicle-to-Everything (V2X) channels. This paper gives a thorough study of this subject, by first modifying the classical approaches, e.g., Extended Kalman Filter (EKF) and Particle Filter (PF), for non-stationary scenarios, and then proposing a Reinforcement Learning (RL)-based approach that can achieve the URLLC requirements in a typical intersection scenario. Simulation results based on a commercial ray-tracing simulator show that enhanced EKF and PF methods achieve packet delay more than 10 ms, whereas the proposed deep RL-based method can reduce the latency to about 6 ms, by extracting context information from the training data.