# BERT4beam: Large AI Model Enabled Generalized Beamforming Optimization

arXiv: https://arxiv.org/abs/2509.11056 · HTML: https://mlsanity.com/paper/2509.11056/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 14 Sep 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [The Roadmap to 6G -- AI Empowered Wireless Networks](https://mlsanity.com/paper/1904.11686/) (2019)
- [Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management](https://mlsanity.com/paper/1705.09412/) (2017)
- [Graph Neural Networks for Scalable Radio Resource Management: Architecture Design and Theoretical Analysis](https://mlsanity.com/paper/2007.07632/) (2020)
- [Large Language Models Empowered Autonomous Edge AI for Connected Intelligence](https://mlsanity.com/paper/2307.02779/) (2023)
- [Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization](https://mlsanity.com/paper/2308.05384/) (2023)
- [Graph Neural Networks for Wireless Communications: From Theory to Practice](https://mlsanity.com/paper/2203.10800/) (2022)
- [LLM4CP: Adapting Large Language Models for Channel Prediction](https://mlsanity.com/paper/2406.14440/) (2024)
- [Large AI Model-Based Semantic Communications](https://mlsanity.com/paper/2307.03492/) (2023)
- [Generative AI Agents with Large Language Model for Satellite Networks via a Mixture of Experts Transmission](https://mlsanity.com/paper/2404.09134/) (2024)
- [Interactive AI with Retrieval-Augmented Generation for Next Generation Networking](https://mlsanity.com/paper/2401.11391/) (2024)
- [Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systems](https://mlsanity.com/paper/2402.01748/) (2024)
- [Beam Prediction based on Large Language Models](https://mlsanity.com/paper/2408.08707/) (2024)
- [AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems](https://mlsanity.com/paper/2311.05842/) (2023)
- [A Survey of Recent Advances in Optimization Methods for Wireless Communications](https://mlsanity.com/paper/2401.12025/) (2024)
- [LLM4WM: Adapting LLM for Wireless Multi-Tasking](https://mlsanity.com/paper/2501.12983/) (2025)
- [Scalable Power Control/Beamforming in Heterogeneous Wireless Networks with Graph Neural Networks](https://mlsanity.com/paper/2104.05463/) (2021)
- [Large Language Model Enabled Multi-Task Physical Layer Network](https://mlsanity.com/paper/2412.20772/) (2024)
- [Deep Learning Methods for Universal MISO Beamforming](https://mlsanity.com/paper/2007.00841/) (2020)
- [Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directions](https://mlsanity.com/paper/2407.04581/) (2024)
- [Toward Democratized Generative AI in Next-Generation Mobile Edge Networks](https://mlsanity.com/paper/2411.09148/) (2024)
- [GNN-Based Beamforming for Sum-Rate Maximization in MU-MISO Networks](https://mlsanity.com/paper/2311.03659/) (2023)
- [Prompt-Enabled Large AI Models for CSI Feedback](https://mlsanity.com/paper/2501.10629/) (2025)
- [Agentic Graph Neural Networks for Wireless Communications and Networking Towards Edge General Intelligence: A Survey](https://mlsanity.com/paper/2508.08620/) (2025)
- [Learning Robust Beamforming for MISO Downlink Systems](https://mlsanity.com/paper/2103.01602/) (2021)
- [BeamLLM: Vision-Empowered mmWave Beam Prediction with Large Language Models](https://mlsanity.com/paper/2503.10432/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Semi-amortized Lifted Learning-to-Optimize Masked (SALLO-M) Transformer Model for Scalable and Generalizable Beamforming](https://mlsanity.com/paper/2510.13077/) (2025)

## Abstract (arXiv)

Artificial intelligence (AI) is anticipated to emerge as a pivotal enabler for the forthcoming sixth-generation (6G) wireless communication systems. However, current research efforts regarding large AI models for wireless communications primarily focus on fine-tuning pre-trained large language models (LLMs) for specific tasks. This paper investigates the large-scale AI model designed for beamforming optimization to adapt and generalize to diverse tasks defined by system utilities and scales. We propose a novel framework based on bidirectional encoder representations from transformers (BERT), termed BERT4beam. We aim to formulate the beamforming optimization problem as a token-level sequence learning task, perform tokenization of the channel state information, construct the BERT model, and conduct task-specific pre-training and fine-tuning strategies. Based on the framework, we propose two BERT-based approaches for single-task and multi-task beamforming optimization, respectively. Both approaches are generalizable for varying user scales. Moreover, the former can adapt to varying system utilities and antenna configurations by re-configuring the input and output module of the BERT model, while the latter, termed UBERT, can directly generalize to diverse tasks, due to a finer-grained tokenization strategy. Extensive simulation results demonstrate that the two proposed approaches can achieve near-optimal performance and outperform existing AI models across various beamforming optimization tasks, showcasing strong adaptability and generalizability.