# E2E Learning Massive MIMO for Multimodal Semantic Non-Orthogonal Transmission and Fusion

arXiv: https://arxiv.org/abs/2509.19312 · HTML: https://mlsanity.com/paper/2509.19312/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [UniT: Multimodal Multitask Learning with a Unified Transformer](https://mlsanity.com/paper/2102.10772/) (2021)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Learning to Optimize: Training Deep Neural Networks for Wireless Resource Management](https://mlsanity.com/paper/1705.09412/) (2017)
- [Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges](https://mlsanity.com/paper/2207.00427/) (2022)
- [Nonlinear Transform Source-Channel Coding for Semantic Communications](https://mlsanity.com/paper/2112.10961/) (2021)
- [Wireless Image Retrieval at the Edge](https://mlsanity.com/paper/2007.10915/) (2020)
- [CDDM: Channel Denoising Diffusion Models for Wireless Semantic Communications](https://mlsanity.com/paper/2309.08895/) (2023)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Task-Oriented Multi-User Semantic Communications for VQA Task](https://mlsanity.com/paper/2108.07357/) (2021)
- [Wireless Deep Video Semantic Transmission](https://mlsanity.com/paper/2205.13129/) (2022)
- [DeepWiVe: Deep-Learning-Aided Wireless Video Transmission](https://mlsanity.com/paper/2111.13034/) (2021)
- [Deep Learning for Distributed Channel Feedback and Multiuser Precoding in FDD Massive MIMO](https://mlsanity.com/paper/2007.06512/) (2020)
- [OFDM-guided Deep Joint Source Channel Coding for Wireless Multipath Fading Channels](https://mlsanity.com/paper/2109.05194/) (2021)
- [Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communication](https://mlsanity.com/paper/2205.03770/) (2022)
- [Distributed Deep Joint Source-Channel Coding over a Multiple Access Channel](https://mlsanity.com/paper/2211.09920/) (2022)
- [Compressive Sensing Based Adaptive Active User Detection and Channel Estimation: Massive Access Meets Massive MIMO](https://mlsanity.com/paper/1906.09867/) (2019)
- [Deep Learning based Denoise Network for CSI Feedback in FDD Massive MIMO Systems](https://mlsanity.com/paper/2004.07576/) (2020)
- [Data-Driven Deep Learning to Design Pilot and Channel Estimator For Massive MIMO](https://mlsanity.com/paper/2003.05875/) (2020)
- [Model-Driven Deep Learning Based Channel Estimation and Feedback for Millimeter-Wave Massive Hybrid MIMO Systems](https://mlsanity.com/paper/2104.11052/) (2021)
- [Prospective Multiple Antenna Technologies for Beyond 5G](https://mlsanity.com/paper/1910.00092/) (2019)
- [Robust Image Semantic Coding with Learnable CSI Fusion Masking over MIMO Fading Channels](https://mlsanity.com/paper/2406.07389/) (2024)
- [Vision Transformer for Adaptive Image Transmission over MIMO Channels](https://mlsanity.com/paper/2210.15347/) (2022)
- [Toward Adaptive Semantic Communications: Efficient Data Transmission via Online Learned Nonlinear Transform Source-Channel Coding](https://mlsanity.com/paper/2211.04339/) (2022)
- [Data-Driven Deep Learning Based Hybrid Beamforming for Aerial Massive MIMO-OFDM Systems with Implicit CSI](https://mlsanity.com/paper/2201.06778/) (2022)
- [Compressive Sensing-Based Grant-Free Massive Access for 6G Massive Communication](https://mlsanity.com/paper/2311.06770/) (2023)
- [End-to-End Learning for Task-Oriented Semantic Communications Over MIMO Channels: An Information-Theoretic Framework](https://mlsanity.com/paper/2408.17397/) (2024)
- [Deep Learning for Channel Sensing and Hybrid Precoding in TDD Massive MIMO OFDM Systems](https://mlsanity.com/paper/2011.10709/) (2020)
- [Next-Generation URLLC with Massive Devices: A Unified Semi-Blind Detection Framework for Sourced and Unsourced Random Access](https://mlsanity.com/paper/2303.04414/) (2023)

## Abstract (arXiv)

This paper investigates multimodal semantic non-orthogonal transmission and fusion in hybrid analog-digital massive multiple-input multiple-output (MIMO). A Transformer-based cross-modal source-channel semantic-aware network (CSC-SA-Net) framework is conceived, where channel state information (CSI) reference signal (RS), feedback, analog-beamforming/combining, and baseband semantic processing are data-driven end-to-end (E2E) optimized at the base station (BS) and user equipments (UEs). CSC-SA-Net comprises five sub-networks: BS-side CSI-RS network (BS-CSIRS-Net), UE-side channel semantic-aware network (UE-CSANet), BS-CSANet, UE-side multimodal semantic fusion network (UE-MSFNet), and BS-MSFNet. Specifically, we firstly E2E train BS-CSIRS-Net, UE-CSANet, and BS-CSANet to jointly design CSI-RS, feedback, analog-beamforming/combining with maximum physical-layer's spectral-efficiency. Meanwhile, we E2E train UE-MSFNet and BS-MSFNet for optimizing application-layer's source semantic downstream tasks. On these pre-trained models, we further integrate application-layer semantic processing with physical-layer tasks to E2E train five subnetworks. Extensive simulations show that the proposed CSC-SA-Net outperforms traditional separated designs, revealing the advantage of cross-modal channel-source semantic fusion.