# CSI-GPT: Integrating Generative Pre-Trained Transformer with Federated-Tuning to Acquire Downlink Massive MIMO Channels

arXiv: https://arxiv.org/abs/2406.03438 · HTML: https://mlsanity.com/paper/2406.03438/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 21 of 21)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [Federated Learning over Wireless Fading Channels](https://mlsanity.com/paper/1907.09769/) (2019)
- [Communication-Efficient Adaptive Federated Learning](https://mlsanity.com/paper/2205.02719/) (2022)
- [Overview of Deep Learning-based CSI Feedback in Massive MIMO Systems](https://mlsanity.com/paper/2206.14383/) (2022)
- [FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers](https://mlsanity.com/paper/2211.08025/) (2022)
- [Federated Learning for Channel Estimation in Conventional and RIS-Assisted Massive MIMO](https://mlsanity.com/paper/2008.10846/) (2020)
- [Transformer-Empowered 6G Intelligent Networks: From Massive MIMO Processing to Semantic Communication](https://mlsanity.com/paper/2205.03770/) (2022)
- [Pruning the Pilots: Deep Learning-Based Pilot Design and Channel Estimation for MIMO-OFDM Systems](https://mlsanity.com/paper/2006.11796/) (2020)
- [Integrated Sensing and Communication with Massive MIMO: A Unified Tensor Approach for Channel and Target Parameter Estimation](https://mlsanity.com/paper/2401.01738/) (2024)
- [Deep Learning for Joint Channel Estimation and Feedback in Massive MIMO Systems](https://mlsanity.com/paper/2011.07242/) (2020)
- [Massive Digital Over-the-Air Computation for Communication-Efficient Federated Edge Learning](https://mlsanity.com/paper/2405.15969/) (2024)
- [Data-Driven Deep Learning to Design Pilot and Channel Estimator For Massive MIMO](https://mlsanity.com/paper/2003.05875/) (2020)
- [Model-Driven Deep Learning Based Channel Estimation and Feedback for Millimeter-Wave Massive Hybrid MIMO Systems](https://mlsanity.com/paper/2104.11052/) (2021)
- [Bit-level Optimized Neural Network for Multi-antenna Channel Quantization](https://mlsanity.com/paper/1909.10730/) (2019)
- [CAnet: Uplink-aided Downlink Channel Acquisition in FDD Massive MIMO using Deep Learning](https://mlsanity.com/paper/2101.04377/) (2021)
- [Machine Learning-Based CSI Feedback With Variable Length in FDD Massive MIMO](https://mlsanity.com/paper/2204.04723/) (2022)
- [Near-Space Communications: the Last Piece of 6G Space-Air-Ground-Sea Integrated Network Puzzle](https://mlsanity.com/paper/2401.00283/) (2023)
- [Lightweight Neural Network with Knowledge Distillation for CSI Feedback](https://mlsanity.com/paper/2210.17113/) (2022)
- [Fluid Antenna-Assisted MIMO Transmission Exploiting Statistical CSI](https://mlsanity.com/paper/2312.07895/) (2023)
- [Near-Field Wideband Extremely Large-scale MIMO Transmission with Holographic Metasurface Antennas](https://mlsanity.com/paper/2205.02533/) (2022)

## Built on by (most cited first, 5 of 5)

- [WirelessGPT: A Generative Pre-trained Multi-task Learning Framework for Wireless Communication](https://mlsanity.com/paper/2502.06877/) (2025)
- [XAInomaly: Explainable and Interpretable Deep Contractive Autoencoder for O-RAN Traffic Anomaly Detection](https://mlsanity.com/paper/2502.09194/) (2025)
- [Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey](https://mlsanity.com/paper/2601.03181/) (2026)
- [Large AI Model for Delay-Doppler Domain Channel Prediction in 6G OTFS-Based Vehicular Networks](https://mlsanity.com/paper/2503.01116/) (2025)
- [Uplink Assisted Joint Channel Estimation and CSI Feedback: An Approach Based on Deep Joint Source-Channel Coding](https://mlsanity.com/paper/2504.10836/) (2025)

## Abstract (arXiv)

In massive multiple-input multiple-output (MIMO) systems, how to reliably acquire downlink channel state information (CSI) with low overhead is challenging. In this work, by integrating the generative pre-trained Transformer (GPT) with federated-tuning, we propose a CSI-GPT approach to realize efficient downlink CSI acquisition. Specifically, we first propose a Swin Transformer-based channel acquisition network (SWTCAN) to acquire downlink CSI, where pilot signals, downlink channel estimation, and uplink CSI feedback are jointly designed. Furthermore, to solve the problem of insufficient training data, we propose a variational auto-encoder-based channel sample generator (VAE-CSG), which can generate sufficient CSI samples based on a limited number of high-quality CSI data obtained from the current cell. The CSI dataset generated from VAE-CSG will be used for pre-training SWTCAN. To fine-tune the pre-trained SWTCAN for improved performance, we propose an online federated-tuning method, where only a small amount of SWTCAN parameters are unfrozen and updated using over-the-air computation, avoiding the high communication overhead caused by aggregating the complete CSI samples from user equipment (UEs) to the BS for centralized fine-tuning. Simulation results verify the advantages of the proposed SWTCAN and the communication efficiency of the proposed federated-tuning method. Our code is publicly available at https://github.com/BIT-ZY/CSI-GPT