# Multi-Modal Transformer and Reinforcement Learning-based Beam Management

arXiv: https://arxiv.org/abs/2410.19859 · HTML: https://mlsanity.com/paper/2410.19859/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 22 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 8 of 8)

- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets](https://mlsanity.com/paper/2111.07574/) (2021)
- [Multi-Modal Beam Prediction Challenge 2022: Towards Generalization](https://mlsanity.com/paper/2209.07519/) (2022)
- [Multimodal Transformers for Wireless Communications: A Case Study in Beam Prediction](https://mlsanity.com/paper/2309.11811/) (2023)
- [Radio Resource and Beam Management in 5G mmWave Using Clustering and Deep Reinforcement Learning](https://mlsanity.com/paper/2103.05047/) (2021)
- [Joint Sensing and Communications for Deep Reinforcement Learning-based Beam Management in 6G](https://mlsanity.com/paper/2208.01880/) (2022)
- [Knowledge Transfer based Radio and Computation Resource Allocation for 5G RAN Slicing](https://mlsanity.com/paper/2110.12245/) (2021)

## Built on by (most cited first, 8 of 8)

- [When IoT Meet LLMs: Applications and Challenges](https://mlsanity.com/paper/2411.17722/) (2024)
- [M2BeamLLM: Multimodal Sensing-empowered mmWave Beam Prediction with Large Language Models](https://mlsanity.com/paper/2506.14532/) (2025)
- [Beam Selection in ISAC using Contextual Bandit with Multi-modal Transformer and Transfer Learning](https://mlsanity.com/paper/2503.08937/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Generative AI Meets Wireless Sensing: Towards Wireless Foundation Model](https://mlsanity.com/paper/2509.15258/) (2025)
- [Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework](https://mlsanity.com/paper/2602.13606/) (2026)
- [Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networks](https://mlsanity.com/paper/2606.05208/) (2026)
- [WiSDoM: Wireless Sparse Decision Transformer with Mixture-of-Experts for Multi-Task Mobile Network Optimization](https://mlsanity.com/paper/2609.00284/) (2026)

## Abstract (arXiv)

Beam management is an important technique to improve signal strength and reduce interference in wireless communication systems. Recently, there has been increasing interest in using diverse sensing modalities for beam management. However, it remains a big challenge to process multi-modal data efficiently and extract useful information. On the other hand, the recently emerging multi-modal transformer (MMT) is a promising technique that can process multi-modal data by capturing long-range dependencies. While MMT is highly effective in handling multi-modal data and providing robust beam management, integrating reinforcement learning (RL) further enhances their adaptability in dynamic environments. In this work, we propose a two-step beam management method by combining MMT with RL for dynamic beam index prediction. In the first step, we divide available beam indices into several groups and leverage MMT to process diverse data modalities to predict the optimal beam group. In the second step, we employ RL for fast beam decision-making within each group, which in return maximizes throughput. Our proposed framework is tested on a 6G dataset. In this testing scenario, it achieves higher beam prediction accuracy and system throughput compared to both the MMT-only based method and the RL-only based method.