# Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction

arXiv: https://arxiv.org/abs/1911.06255 · HTML: https://mlsanity.com/paper/1911.06255/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2019.

## Built on by (most cited first, 29 of 29)

- [Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff](https://mlsanity.com/paper/2102.09527/) (2021)
- [Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network](https://mlsanity.com/paper/2201.04712/) (2022)
- [Federated mmWave Beam Selection Utilizing LIDAR Data](https://mlsanity.com/paper/2102.02802/) (2021)
- [Real-time Outdoor Localization Using Radio Maps: A Deep Learning Approach](https://mlsanity.com/paper/2106.12556/) (2021)
- [LocUNet: Fast Urban Positioning Using Radio Maps and Deep Learning](https://mlsanity.com/paper/2202.00738/) (2022)
- [Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios](https://mlsanity.com/paper/2308.06868/) (2023)
- [Deep Learning Assisted Calibrated Beam Training for Millimeter-Wave Communication Systems](https://mlsanity.com/paper/2101.05206/) (2021)
- [LIDAR and Position-Aided mmWave Beam Selection with Non-local CNNs and Curriculum Training](https://mlsanity.com/paper/2104.14579/) (2021)
- [VOMTC: Vision Objects for Millimeter and Terahertz Communications](https://mlsanity.com/paper/2409.09330/) (2024)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Vision-Aided Beam Tracking: Explore the Proper Use of Camera Images with Deep Learning](https://mlsanity.com/paper/2109.14686/) (2021)
- [ViT LoS V2X: Vision Transformers for Environment-aware LoS Blockage Prediction for 6G Vehicular Networks](https://mlsanity.com/paper/2407.15023/) (2024)
- [Multi-modal Image and Radio Frequency Fusion for Optimizing Vehicle Positioning](https://mlsanity.com/paper/2410.19788/) (2024)
- [Multi-modal Data based Semi-Supervised Learning for Vehicle Positioning](https://mlsanity.com/paper/2410.20680/) (2024)
- [Multimodal Mixture-of-Experts for ISAC in Low-Altitude Wireless Networks](https://mlsanity.com/paper/2512.01750/) (2025)
- [Intelligent Blockage Prediction and Proactive Handover for Seamless Connectivity in Vision-Aided 5G/6G UDNs](https://mlsanity.com/paper/2203.16419/) (2022)
- [Wireless Channel Prediction in Partially Observed Environments](https://mlsanity.com/paper/2207.00934/) (2022)
- [Overhead-Free Blockage Detection and Precoding Through Physics-Based Graph Neural Networks: LIDAR Data Meets Ray Tracing](https://mlsanity.com/paper/2209.07350/) (2022)
- [Outage Performance and Novel Loss Function for an ML-Assisted Resource Allocation: An Exact Analytical Framework](https://mlsanity.com/paper/2305.09739/) (2023)
- [Vision-Assisted Digital Twin Creation for mmWave Beam Management](https://mlsanity.com/paper/2401.17781/) (2024)
- [Vision Aided Channel Prediction for Vehicular Communications: A Case Study of Received Power Prediction Using RGB Images](https://mlsanity.com/paper/2501.18618/) (2025)
- [Deep Learning for Moving Blockage Prediction using Real Millimeter Wave Measurements](https://mlsanity.com/paper/2101.06886/) (2021)
- [Rotate the ReLU to implicitly sparsify deep networks](https://mlsanity.com/paper/2206.00488/) (2022)
- [Causality-Driven Reinforcement Learning for Joint Communication and Sensing](https://mlsanity.com/paper/2409.15329/) (2024)
- [CONVERGE: A Multi-Agent Vision-Radio Architecture for xApps](https://mlsanity.com/paper/2508.04556/) (2025)
- [Enwar 3.0: An Agentic Multi-Modal LLM Orchestrator for Situation-Aware Beamforming, Blockage Prediction, and Handover Management](https://mlsanity.com/paper/2605.03215/) (2026)
- [Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivity](https://mlsanity.com/paper/2605.05071/) (2026)
- [RIS-Assisted Proactive Handover for Reliable mmWave Wireless Networks](https://mlsanity.com/paper/2606.26885/) (2026)
- [Vision-Wireless Fusion for Multi-User Localization: A Cross-Modal Transformer Approach](https://mlsanity.com/paper/2609.23372/) (2026)

## Abstract (arXiv)

This paper investigates a novel research direction that leverages vision to help overcome the critical wireless communication challenges. In particular, this paper considers millimeter wave (mmWave) communication systems, which are principal components of 5G and beyond. These systems face two important challenges: (i) the large training overhead associated with selecting the optimal beam and (ii) the reliability challenge due to the high sensitivity to link blockages. Interestingly, most of the devices that employ mmWave arrays will likely also use cameras, such as 5G phones, self-driving vehicles, and virtual/augmented reality headsets. Therefore, we investigate the potential gains of employing cameras at the mmWave base stations and leveraging their visual data to help overcome the beam selection and blockage prediction challenges. To do that, this paper exploits computer vision and deep learning tools to predict mmWave beams and blockages directly from the camera RGB images and the sub-6GHz channels. The experimental results reveal interesting insights into the effectiveness of such solutions. For example, the deep learning model is capable of achieving over 90% beam prediction accuracy, which only requires snapping a shot of the scene and zero overhead.