# BeamLLM: Vision-Empowered mmWave Beam Prediction with Large Language Models

arXiv: https://arxiv.org/abs/2503.10432 · HTML: https://mlsanity.com/paper/2503.10432/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 16 references that are in the map.
- First posted on arXiv 13 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [Time-LLM: Time Series Forecasting by Reprogramming Large Language Models](https://mlsanity.com/paper/2310.01728/) (2023)
- [CAT: Cross Attention in Vision Transformer](https://mlsanity.com/paper/2106.05786/) (2021)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [LLM4CP: Adapting Large Language Models for Channel Prediction](https://mlsanity.com/paper/2406.14440/) (2024)
- [LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications](https://mlsanity.com/paper/2203.05548/) (2022)
- [Beam Prediction based on Large Language Models](https://mlsanity.com/paper/2408.08707/) (2024)
- [Radar Aided 6G Beam Prediction: Deep Learning Algorithms and Real-World Demonstration](https://mlsanity.com/paper/2111.09676/) (2021)
- [Position Aided Beam Prediction in the Real World: How Useful GPS Locations Actually Are?](https://mlsanity.com/paper/2205.09054/) (2022)
- [Computer Vision Aided Beam Tracking in A Real-World Millimeter Wave Deployment](https://mlsanity.com/paper/2111.14803/) (2021)
- [Learning to Estimate RIS-Aided mmWave Channels](https://mlsanity.com/paper/2107.12631/) (2021)
- [Port-LLM: A Port Prediction Method for Fluid Antenna based on Large Language Models](https://mlsanity.com/paper/2502.09857/) (2025)

## Built on by (most cited first, 5 of 5)

- [LVM4CSI: Enabling Direct Application of Pre-Trained Large Vision Models for Wireless Channel Tasks](https://mlsanity.com/paper/2507.05121/) (2025)
- [M2BeamLLM: Multimodal Sensing-empowered mmWave Beam Prediction with Large Language Models](https://mlsanity.com/paper/2506.14532/) (2025)
- [BERT4beam: Large AI Model Enabled Generalized Beamforming Optimization](https://mlsanity.com/paper/2509.11056/) (2025)
- [Vehicle-to-Infrastructure Collaborative Spatial Perception via Multimodal Large Language Models](https://mlsanity.com/paper/2509.03837/) (2025)
- [CM-MAE: A Physics-Guided Cross-Modal Self-Supervised Learning Framework for Vision-Wireless Applications](https://mlsanity.com/paper/2608.15972/) (2026)

## Abstract (arXiv)

In this paper, we propose BeamLLM, a vision-aided millimeter-wave (mmWave) beam prediction framework leveraging large language models (LLMs) to address the challenges of high training overhead and latency in mmWave communication systems. By combining computer vision (CV) with LLMs' cross-modal reasoning capabilities, the framework extracts user equipment (UE) positional features from RGB images and aligns visual-temporal features with LLMs' semantic space through reprogramming techniques. Evaluated on a realistic vehicle-to-infrastructure (V2I) scenario, the proposed method achieves 61.01% top-1 accuracy and 97.39% top-3 accuracy in standard prediction tasks, significantly outperforming traditional deep learning models. In few-shot prediction scenarios, the performance degradation is limited to 12.56% (top-1) and 5.55% (top-3) from time sample 1 to 10, demonstrating superior prediction capability.