# Deep Multimodal Learning: Merging Sensory Data for Massive MIMO Channel Prediction

arXiv: https://arxiv.org/abs/2007.09366 · HTML: https://mlsanity.com/paper/2007.09366/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [CSI-BERT2: A BERT-inspired Framework for Efficient CSI Prediction and Classification in Wireless Communication and Sensing](https://mlsanity.com/paper/2412.06861/) (2024)
- [CAnet: Uplink-aided Downlink Channel Acquisition in FDD Massive MIMO using Deep Learning](https://mlsanity.com/paper/2101.04377/) (2021)
- [SIMAC: A Semantic-Driven Integrated Multimodal Sensing And Communication Framework](https://mlsanity.com/paper/2503.08726/) (2025)
- [Differentially Private Federated Learning via Reconfigurable Intelligent Surface](https://mlsanity.com/paper/2203.17028/) (2022)
- [Large Language Model-Driven Distributed Integrated Multimodal Sensing and Semantic Communications](https://mlsanity.com/paper/2505.18194/) (2025)
- [Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems](https://mlsanity.com/paper/2506.22374/) (2025)
- [CSI-tuples-based 3D Channel Fingerprints Construction Assisted by MultiModal Learning](https://mlsanity.com/paper/2603.25288/) (2026)

## Abstract (arXiv)

Existing work in intelligent communications has recently made preliminary attempts to utilize multi-source sensing information (MSI) to improve the system performance. However, the research on MSI aided intelligent communications has not yet explored how to integrate and fuse the multimodal sensory data, which motivates us to develop a systematic framework for wireless communications based on deep multimodal learning (DML). In this paper, we first present complete descriptions and heuristic understandings on the framework of DML based wireless communications, where core design choices are analyzed in the view of communications. Then, we develop several DML based architectures for channel prediction in massive multiple-input multiple-output (MIMO) systems that leverage various modality combinations and fusion levels. The case study of massive MIMO channel prediction offers an important example that can be followed in developing other DML based communication technologies. Simulations results demonstrate that the proposed DML framework can effectively exploit the constructive and complementary information of multimodal sensory data in various wireless communication scenarios.