# M2BeamLLM: Multimodal Sensing-empowered mmWave Beam Prediction with Large Language Models

arXiv: https://arxiv.org/abs/2506.14532 · HTML: https://mlsanity.com/paper/2506.14532/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Multimodal Alignment and Fusion: A Survey](https://mlsanity.com/paper/2411.17040/) (2024)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [Deep Learning for mmWave Beam and Blockage Prediction Using Sub-6GHz Channels](https://mlsanity.com/paper/1910.02900/) (2019)
- [LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications](https://mlsanity.com/paper/2203.05548/) (2022)
- [A Survey of Beam Management for mmWave and THz Communications Towards 6G](https://mlsanity.com/paper/2308.02135/) (2023)
- [Radar Aided 6G Beam Prediction: Deep Learning Algorithms and Real-World Demonstration](https://mlsanity.com/paper/2111.09676/) (2021)
- [Position Aided Beam Prediction in the Real World: How Useful GPS Locations Actually Are?](https://mlsanity.com/paper/2205.09054/) (2022)
- [Computer Vision Aided Beam Tracking in A Real-World Millimeter Wave Deployment](https://mlsanity.com/paper/2111.14803/) (2021)
- [Multi-Modal Transformer and Reinforcement Learning-based Beam Management](https://mlsanity.com/paper/2410.19859/) (2024)
- [BeamLLM: Vision-Empowered mmWave Beam Prediction with Large Language Models](https://mlsanity.com/paper/2503.10432/) (2025)
- [Multimodal Deep Learning-Empowered Beam Prediction in Future THz ISAC Systems](https://mlsanity.com/paper/2505.02381/) (2025)

## Built on by (most cited first, 4 of 4)

- [Large AI Models for Wireless Physical Layer](https://mlsanity.com/paper/2508.02314/) (2025)
- [Wireless Power Control Based on Large Language Models](https://mlsanity.com/paper/2603.00474/) (2026)
- [Empower Low-Altitude Economy: A Reliability-Aware Dynamic Weighting Allocation for Multi-modal UAV Beam Prediction](https://mlsanity.com/paper/2512.24324/) (2025)
- [Structure-Aware Multimodal LLM Framework for Trustworthy Near-Field Beam Prediction](https://mlsanity.com/paper/2603.16143/) (2026)

## Abstract (arXiv)

This paper introduces a novel neural network framework called M2BeamLLM for beam prediction in millimeter-wave (mmWave) massive multi-input multi-output (mMIMO) communication systems. M2BeamLLM integrates multi-modal sensor data, including images, radar, LiDAR, and GPS, leveraging the powerful reasoning capabilities of large language models (LLMs) such as GPT-2 for beam prediction. By combining sensing data encoding, multimodal alignment and fusion, and supervised fine-tuning (SFT), M2BeamLLM achieves significantly higher beam prediction accuracy and robustness, demonstrably outperforming traditional deep learning (DL) models in both standard and few-shot scenarios. Furthermore, its prediction performance consistently improves with increased diversity in sensing modalities. Our study provides an efficient and intelligent beam prediction solution for vehicle-to-infrastructure (V2I) mmWave communication systems.