# Sensing and Understanding the World over Air: A Large Multimodal Model for Mobile Networks

arXiv: https://arxiv.org/abs/2511.21707 · HTML: https://mlsanity.com/paper/2511.21707/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 12 of its 15 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 12 of 12)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [Timer: Generative Pre-trained Transformers Are Large Time Series Models](https://mlsanity.com/paper/2402.02368/) (2024)
- [NetLLM: Adapting Large Language Models for Networking](https://mlsanity.com/paper/2402.02338/) (2024)
- [DeepSense 6G: A Large-Scale Real-World Multi-Modal Sensing and Communication Dataset](https://mlsanity.com/paper/2211.09769/) (2022)
- [LLM4CP: Adapting Large Language Models for Channel Prediction](https://mlsanity.com/paper/2406.14440/) (2024)
- [Beam Prediction based on Large Language Models](https://mlsanity.com/paper/2408.08707/) (2024)
- [Generative AI for Physical Layer Communications: A Survey](https://mlsanity.com/paper/2312.05594/) (2023)
- [Intelligent Multi-Modal Sensing-Communication Integration: Synesthesia of Machines](https://mlsanity.com/paper/2306.14143/) (2023)
- [Finding the Missing Data: A BERT-inspired Approach Against Package Loss in Wireless Sensing](https://mlsanity.com/paper/2403.12400/) (2024)
- [LLM4WM: Adapting LLM for Wireless Multi-Tasking](https://mlsanity.com/paper/2501.12983/) (2025)

## Abstract (arXiv)

Large models (LMs), such as ChatGPT, have made a significant impact across diverse domains and hold great potential to facilitate the evolution of network intelligence. Wireless-native multi-modal large models (WMLMs) can sense and understand the physical world through multi-modal data, serving as a key enabler that integrates communication, sensing, and intelligence, and thus they can boost various smart services to billions of users. However, research on WMLMs remains in its infancy, and the construction of domain-specific multi-modal large models for wireless networks is still underexplored. In this paper, we outlines the key characteristics of WMLMs and summarizes existing methods, on the basis of which a wireless-native multimodal training paradigm is proposed. Specifically, we constructed a GPT-style WMLM model and trained it on a real-world large-scale dataset, leveraging wireless signals as an anchor modality for contrastive learning. Our approach demonstrates outstanding performance compared with existing small-scale models and large multi-modal models, validating the feasibility of using wireless signals as a universal modality and highlighting WMLM's potential to emerge as a new paradigm for future wireless networks.