# WALoMA: A Multitask Wireless Foundation Model via Adaptive Low-Rank Masked Autoencoders

arXiv: https://arxiv.org/abs/2607.25763 · HTML: https://mlsanity.com/paper/2607.25763/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 28 Jul 2026.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DeepMIMO: A Generic Deep Learning Dataset for Millimeter Wave and Massive MIMO Applications](https://mlsanity.com/paper/1902.06435/) (2019)
- [Large Wireless Model (LWM): A Foundation Model for Wireless Channels](https://mlsanity.com/paper/2411.08872/) (2024)
- [WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication](https://mlsanity.com/paper/2502.06877/) (2025)
- [Convolutional Neural Network based Multiple-Rate Compressive Sensing for Massive MIMO CSI Feedback: Design, Simulation, and Analysis](https://mlsanity.com/paper/1906.06007/) (2019)
- [A Multi-Task Foundation Model for Wireless Channel Representation Using Contrastive and Masked Autoencoder Learning](https://mlsanity.com/paper/2505.09160/) (2025)
- [LLM4WM: Adapting LLM for Wireless Multi-Tasking](https://mlsanity.com/paper/2501.12983/) (2025)
- [Lightweight Convolutional Neural Networks for CSI Feedback in Massive MIMO](https://mlsanity.com/paper/2005.00438/) (2020)
- [Large Language Model Enabled Multi-Task Physical Layer Network](https://mlsanity.com/paper/2412.20772/) (2024)
- [ENWAR: A RAG-empowered Multi-Modal LLM Framework for Wireless Environment Perception](https://mlsanity.com/paper/2410.18104/) (2024)
- [Deep Learning in Physical Layer: Review on Data Driven End-to-End Communication Systems and their Enabling Semantic Applications](https://mlsanity.com/paper/2401.12800/) (2024)
- [Exploring the 6G Potentials: Immersive, Hyper Reliable, and Low-Latency Communication](https://mlsanity.com/paper/2407.11051/) (2024)
- [A Multi-Modal Foundational Model for Wireless Communication and Sensing](https://mlsanity.com/paper/2602.04016/) (2026)
- [Cross Far- and Near-Field Beam Management Technologies in Millimeter-Wave and Terahertz MIMO Systems](https://mlsanity.com/paper/2504.18855/) (2025)

## Abstract (arXiv)

This paper proposes a multitask wireless foundation model via adaptive low-rank masked autoencoders (WALoMA), a unified multi-task foundation model for sixth-generation (6G) wireless physical layer architectures, to address the limitations of specialized, task-specific deep learning models and the practical challenge of scarce labeled wireless datasets. By leveraging concepts inspired by foundation models, the proposed framework adopts a masked autoencoder (MAE) paradigm to learn from unlabeled channel data, to significantly reduce reliance on extensive annotations. The model treats wireless channel state information (CSI) as a universal modality and learns transferable representations through self-supervised channel reconstruction. Key architectural novelties include the use of 2D positional encoding (PE) to explicitly preserve the spatial-frequency relationships between antennas and subcarriers, and low-rank adaptation (LoRA) for parameter-efficient fine-tuning. The framework's efficacy is demonstrated across five downstream tasks, achieving individual scores of 96.47% for LoS/NLoS classification, 80.45% for beam prediction, 85.78% for channel interpolation, 99.12% for channel estimation, and 77.18% for channel charting. Consequently, numerical results show that the proposed model achieves a composite score of 87.80%, significantly outperforming the 59.90% achieved by the large wireless model (LWM) baseline while training an average of only 14.68% of total parameters, and maintaining strong performance even under extremely limited labeled data conditions.