mlsanityData to 14 Oct 2026

Towards a Foundation Model for Communication Systems

arXiv 2505.14603 ↗First posted 20 May 2025cs.AI cs.LG eess.SPcited by 2

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence (AI) has demonstrated unprecedented performance across various domains, and its application to communication systems is an active area of research. While current methods focus on task-specific solutions, the broader trend in AI is shifting toward large general models capable of supporting multiple applications. In this work, we take a step toward a foundation model for communication data--a transformer-based, multi-modal model designed to operate directly on communication data. We propose methodologies to address key challenges, including tokenization, positional embedding, multimodality, variable feature sizes, and normalization. Furthermore, we empirically demonstrate that such a model can successfully estimate multiple features, including transmission rank, selected precoder, Doppler spread, and delay profile.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

25 of 36 references.

References · 25

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  7. 2023A decoder-only foundation model for time-series forecastingcited by 692
  8. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 482
  9. 2024Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Expertscited by 287
  10. 2025Sundial: A Family of Highly Capable Time Series Foundation Modelscited by 180
  11. 2022Sionna: An Open-Source Library for Next-Generation Physical Layer Researchcited by 145
  12. 2024Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Seriescited by 127
13 more
  1. 2024NVLM: Open Frontier-Class Multimodal LLMscited by 125
  2. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 113
  3. 2023Large-scale Training of Foundation Models for Wearable Biosignalscited by 95
  4. 2024Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Expertscited by 95
  5. 2024Scaling Wearable Foundation Modelscited by 63
  6. 2024WirelessLLM: Empowering Large Language Models Towards Wireless Intelligencecited by 44
  7. 2024Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6Gcited by 31
  8. 2024TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specificationscited by 27
  9. 2024Telco-RAG: Navigating the Challenges of Retrieval-Augmented Language Models for Telecommunicationscited by 26
  10. 2022Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communicationcited by 19
  11. 2024Telecom Language Models: Must They Be Large?cited by 10
  12. 2022Role of Deep Learning in Wireless Communicationscited by 6
  13. 2024Machine Learning in Communications: A Road to Intelligent Transmission and Processingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Robust Processing and Learning: Principles, Methods, and Wireless Applicationscited by 0
  2. 2026A Comprehensive Survey of Wireless Foundation Models for AI-Native 6G Networkscited by 0