# Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systems

arXiv: https://arxiv.org/abs/2402.01748 · HTML: https://mlsanity.com/paper/2402.01748/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 30 Jan 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 16 of 16)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning](https://mlsanity.com/paper/2302.02662/) (2023)
- [TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge](https://mlsanity.com/paper/2310.15051/) (2023)
- [Large Generative AI Models for Telecom: The Next Big Thing?](https://mlsanity.com/paper/2306.10249/) (2023)
- [Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2112.08688/) (2021)
- [Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks](https://mlsanity.com/paper/2107.01431/) (2021)
- [LLM Augmented LLMs: Expanding Capabilities through Composition](https://mlsanity.com/paper/2401.02412/) (2024)
- [Large Language Models for Networking: Applications, Enabling Techniques, and Challenges](https://mlsanity.com/paper/2311.17474/) (2023)
- [Neuro-Symbolic Causal Reasoning Meets Signaling Game for Emergent Semantic Communications](https://mlsanity.com/paper/2210.12040/) (2022)
- [Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc](https://mlsanity.com/paper/2308.04445/) (2023)
- [AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems](https://mlsanity.com/paper/2311.05842/) (2023)
- [Towards Causal Foundation Model: on Duality between Causal Inference and Attention](https://mlsanity.com/paper/2310.00809/) (2023)
- [Causal Reasoning: Charting a Revolutionary Course for Next-Generation AI-Native Wireless Networks](https://mlsanity.com/paper/2309.13223/) (2023)

## Built on by (most cited first, 27 of 27)

- [Mobile Edge Intelligence for Large Language Models: A Contemporary Survey](https://mlsanity.com/paper/2407.18921/) (2024)
- [Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunities](https://mlsanity.com/paper/2405.10825/) (2024)
- [WirelessLLM: Empowering Large Language Models Towards Wireless Intelligence](https://mlsanity.com/paper/2405.17053/) (2024)
- [TelecomGPT: A Framework to Build Telecom-Specfic Large Language Models](https://mlsanity.com/paper/2407.09424/) (2024)
- [Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6G](https://mlsanity.com/paper/2405.02336/) (2024)
- [Tele-LLMs: A Series of Specialized Large Language Models for Telecommunications](https://mlsanity.com/paper/2409.05314/) (2024)
- [GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoning](https://mlsanity.com/paper/2402.16631/) (2024)
- [ENWAR: A RAG-empowered Multi-Modal LLM Framework for Wireless Environment Perception](https://mlsanity.com/paper/2410.18104/) (2024)
- [Adaptive Token Merging for Efficient Transformer Semantic Communication at the Edge](https://mlsanity.com/paper/2509.09955/) (2025)
- [Hermes: A Large Language Model Framework on the Journey to Autonomous Networks](https://mlsanity.com/paper/2411.06490/) (2024)
- [A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysis](https://mlsanity.com/paper/2506.12263/) (2025)
- [Bridging Physical and Digital Worlds: Embodied Large AI for Future Wireless Systems](https://mlsanity.com/paper/2506.24009/) (2025)
- [NetOrchLLM: Mastering Wireless Network Orchestration with Large Language Models](https://mlsanity.com/paper/2412.10107/) (2024)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [AI2MMUM: AI-AI Oriented Multi-Modal Universal Model Leveraging Telecom Domain Large Model](https://mlsanity.com/paper/2505.10003/) (2025)
- [From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications](https://mlsanity.com/paper/2505.22311/) (2025)
- [Zero-Trust Foundation Models: A New Paradigm for Secure and Collaborative Artificial Intelligence for Internet of Things](https://mlsanity.com/paper/2505.23792/) (2025)
- [BERT4beam: Large AI Model Enabled Generalized Beamforming Optimization](https://mlsanity.com/paper/2509.11056/) (2025)
- [Future-Proofing Mobile Networks: A Digital Twin Approach to Multi-Signal Management](https://mlsanity.com/paper/2407.15520/) (2024)
- [R-MTLLMF: Resilient Multi-Task Large Language Model Fusion at the Wireless Edge](https://mlsanity.com/paper/2411.18220/) (2024)
- [When can isotropy help adapt LLMs' next word prediction to numerical domains?](https://mlsanity.com/paper/2505.17135/) (2025)
- [Communications-Incentivized Collaborative Reasoning in NetGPT through Agentic Reinforcement Learning](https://mlsanity.com/paper/2602.00766/) (2026)
- [Internet of Everything in the 6G Era: Paradigms, Enablers, Potentials and Future Directions](https://mlsanity.com/paper/2604.25018/) (2026)
- [Enwar 3.0: An Agentic Multi-Modal LLM Orchestrator for Situation-Aware Beamforming, Blockage Prediction, and Handover Management](https://mlsanity.com/paper/2605.03215/) (2026)
- [OmniLoc: A Geometry-Aware Foundation Model for Anchor-Free UE Localization Across Diverse Indoor Environments](https://mlsanity.com/paper/2606.11490/) (2026)
- [A Comprehensive Survey of Wireless Foundation Models for AI-Native 6G Networks](https://mlsanity.com/paper/2608.14694/) (2026)
- [Wireless Foundation Models: State-of-the-Art and Open Challenges](https://mlsanity.com/paper/2609.04707/) (2026)

## Abstract (arXiv)

Large language models (LLMs) and foundation models have been recently touted as a game-changer for 6G systems. However, recent efforts on LLMs for wireless networks are limited to a direct application of existing language models that were designed for natural language processing (NLP) applications. To address this challenge and create wireless-centric foundation models, this paper presents a comprehensive vision on how to design universal foundation models that are tailored towards the deployment of artificial intelligence (AI)-native networks. Diverging from NLP-based foundation models, the proposed framework promotes the design of large multi-modal models (LMMs) fostered by three key capabilities: 1) processing of multi-modal sensing data, 2) grounding of physical symbol representations in real-world wireless systems using causal reasoning and retrieval-augmented generation (RAG), and 3) enabling instructibility from the wireless environment feedback to facilitate dynamic network adaptation thanks to logical and mathematical reasoning facilitated by neuro-symbolic AI. In essence, these properties enable the proposed LMM framework to build universal capabilities that cater to various cross-layer networking tasks and alignment of intents across different domains. Preliminary results from experimental evaluation demonstrate the efficacy of grounding using RAG in LMMs, and showcase the alignment of LMMs with wireless system designs. Furthermore, the enhanced rationale exhibited in the responses to mathematical questions by LMMs, compared to vanilla LLMs, demonstrates the logical and mathematical reasoning capabilities inherent in LMMs. Building on those results, we present a sequel of open questions and challenges for LMMs. We then conclude with a set of recommendations that ignite the path towards LMM-empowered AI-native systems.