# Sheaf-Based Decentralized Multimodal Learning for Next-Generation Wireless Communication Systems

arXiv: https://arxiv.org/abs/2506.22374 · HTML: https://mlsanity.com/paper/2506.22374/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Exploiting Shared Representations for Personalized Federated Learning](https://mlsanity.com/paper/2102.07078/) (2021)
- [A Vision of 6G Wireless Systems: Applications, Trends, Technologies, and Open Research Problems](https://mlsanity.com/paper/1902.10265/) (2019)
- [Federated Learning for Wireless Communications: Motivation, Opportunities and Challenges](https://mlsanity.com/paper/1908.06847/) (2019)
- [Multimodal Federated Learning via Contrastive Representation Ensemble](https://mlsanity.com/paper/2302.08888/) (2023)
- [Tackling Feature and Sample Heterogeneity in Decentralized Multi-Task Learning: A Sheaf-Theoretic Approach](https://mlsanity.com/paper/2502.01145/) (2025)
- [Proactive Received Power Prediction Using Machine Learning and Depth Images for mmWave Networks](https://mlsanity.com/paper/1803.09698/) (2018)
- [Deep Multimodal Learning: Merging Sensory Data for Massive MIMO Channel Prediction](https://mlsanity.com/paper/2007.09366/) (2020)
- [Towards Real-World 6G Drone Communication: Position and Camera Aided Beam Prediction](https://mlsanity.com/paper/2205.12187/) (2022)
- [Distributed Learning over Networks with Graph-Attention-Based Personalization](https://mlsanity.com/paper/2305.13041/) (2023)
- [Proactively Predicting Dynamic 6G Link Blockages Using LiDAR and In-Band Signatures](https://mlsanity.com/paper/2211.09535/) (2022)
- [Distributed Machine Learning for UAV Swarms: Computing, Sensing, and Semantics](https://mlsanity.com/paper/2301.00912/) (2023)
- [Multi-Modality Sensing in mmWave Beamforming for Connected Vehicles Using Deep Learning](https://mlsanity.com/paper/2504.06173/) (2025)
- [Hybrid Beamforming/Combining for Millimeter Wave MIMO: A Machine Learning Approach](https://mlsanity.com/paper/1910.06585/) (2019)
- [Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation](https://mlsanity.com/paper/2410.07662/) (2024)

## Built on by (most cited first, 1 of 1)

- [SheafAlign: A Sheaf-theoretic Framework for Decentralized Multimodal Alignment](https://mlsanity.com/paper/2510.20540/) (2025)

## Abstract (arXiv)

In large-scale communication systems, increasingly complex scenarios require more intelligent collaboration among edge devices collecting various multimodal sensory data to achieve a more comprehensive understanding of the environment and improve decision-making accuracy. However, conventional federated learning (FL) algorithms typically consider unimodal datasets, require identical model architectures, and fail to leverage the rich information embedded in multimodal data, limiting their applicability to real-world scenarios with diverse modalities and varying client capabilities. To address this issue, we propose Sheaf-DMFL, a novel decentralized multimodal learning framework leveraging sheaf theory to enhance collaboration among devices with diverse modalities. Specifically, each client has a set of local feature encoders for its different modalities, whose outputs are concatenated before passing through a task-specific layer. While encoders for the same modality are trained collaboratively across clients, we capture the intrinsic correlations among clients' task-specific layers using a sheaf-based structure. To further enhance learning capability, we propose an enhanced algorithm named Sheaf-DMFL-Att, which tailors the attention mechanism within each client to capture correlations among different modalities. A rigorous convergence analysis of Sheaf-DMFL-Att is provided, establishing its theoretical guarantees. Extensive simulations are conducted on real-world link blockage prediction and mmWave beamforming scenarios, demonstrate the superiority of the proposed algorithms in such heterogeneous wireless communication systems.