# Channelformer: Attention based Neural Solution for Wireless Channel Estimation and Effective Online Training

arXiv: https://arxiv.org/abs/2302.04368 · HTML: https://mlsanity.com/paper/2302.04368/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 60 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [What Do Compressed Deep Neural Networks Forget?](https://mlsanity.com/paper/1911.05248/) (2019)
- [The Roadmap to 6G -- AI Empowered Wireless Networks](https://mlsanity.com/paper/1904.11686/) (2019)
- [Deep Learning-Based Channel Estimation](https://mlsanity.com/paper/1810.05893/) (2018)
- [Multiplying Matrices Without Multiplying](https://mlsanity.com/paper/2106.10860/) (2021)
- [Pruning the Pilots: Deep Learning-Based Pilot Design and Channel Estimation for MIMO-OFDM Systems](https://mlsanity.com/paper/2006.11796/) (2020)
- [5G New Radio: Unveiling the Essentials of the Next Generation Wireless Access Technology](https://mlsanity.com/paper/1806.06898/) (2018)
- [Channel Estimation for Full-Duplex RIS-assisted HAPS Backhauling with Graph Attention Networks](https://mlsanity.com/paper/2010.12004/) (2020)
- [Attention Based Neural Networks for Wireless Channel Estimation](https://mlsanity.com/paper/2204.13465/) (2022)
- [An Attention-Aided Deep Learning Framework for Massive MIMO Channel Estimation](https://mlsanity.com/paper/2108.09430/) (2021)
- [Low Complexity Channel estimation with Neural Network Solutions](https://mlsanity.com/paper/2201.09934/) (2022)
- [AI-Aided Online Adaptive OFDM Receiver: Design and Experimental Results](https://mlsanity.com/paper/1812.06638/) (2018)
- [Deep Learning Based OFDM Channel Estimation Using Frequency-Time Division and Attention Mechanism](https://mlsanity.com/paper/2107.07161/) (2021)
- [OptComNet: Optimized Neural Networks for Low-Complexity Channel Estimation](https://mlsanity.com/paper/2002.10493/) (2020)
- [A Low Complexity Learning-based Channel Estimation for OFDM Systems with Online Training](https://mlsanity.com/paper/2107.06712/) (2021)

## Built on by (most cited first, 14 of 14)

- [Explainable AI for Enhancing Efficiency of DL-based Channel Estimation](https://mlsanity.com/paper/2407.07009/) (2024)
- [Robust Channel Estimation for Optical Wireless Communications Using Neural Network](https://mlsanity.com/paper/2504.02134/) (2025)
- [Belief Information based Deep Channel Estimation for Massive MIMO Systems](https://mlsanity.com/paper/2407.07744/) (2024)
- [Achieving Robust Channel Estimation Neural Networks by Designed Training Data](https://mlsanity.com/paper/2507.12630/) (2025)
- [Invariant Transformation and Resampling based Epistemic-Uncertainty Reduction](https://mlsanity.com/paper/2602.23315/) (2026)
- [Dual-Attention Based 3D Channel Estimation](https://mlsanity.com/paper/2604.01769/) (2026)
- [Physics-Informed Transformer for Multi-Band Channel Frequency Response Reconstruction](https://mlsanity.com/paper/2604.01944/) (2026)
- [SwiftChannel: Algorithm-Hardware Co-Design for Deep Learning-Based 5G Channel Estimation](https://mlsanity.com/paper/2605.01931/) (2026)
- [Lightweight Adaptive Feature Composition for Heterogeneous Downstream Adaptation of Wireless Foundation Models](https://mlsanity.com/paper/2606.10277/) (2026)
- [CeBed: A Benchmark for Deep Data-Driven OFDM Channel Estimation](https://mlsanity.com/paper/2306.13761/) (2023)
- [Learning MMSE Filters for OFDM Channel Estimation: Attention Transformer Gains at Linear Inference](https://mlsanity.com/paper/2506.00452/) (2025)
- [Hybrid Mamba-Attention Neural Architecture for Channel Estimation](https://mlsanity.com/paper/2601.17108/) (2026)
- [Learning During Detection: Continual Learning for Neural OFDM Receivers via DMRS](https://mlsanity.com/paper/2602.20361/) (2026)
- [ReFLEX: Length-Generalizable CSI Denoising for MIMO-OFDM via Relative-Frequency Bias](https://mlsanity.com/paper/2606.00263/) (2026)

## Abstract (arXiv)

In this paper, we propose an encoder-decoder neural architecture (called Channelformer) to achieve improved channel estimation for orthogonal frequency-division multiplexing (OFDM) waveforms in downlink scenarios. The self-attention mechanism is employed to achieve input precoding for the input features before processing them in the decoder. In particular, we implement multi-head attention in the encoder and a residual convolutional neural architecture as the decoder, respectively. We also employ a customized weight-level pruning to slim the trained neural network with a fine-tuning process, which reduces the computational complexity significantly to realize a low complexity and low latency solution. This enables reductions of up to 70% in the parameters, while maintaining an almost identical performance compared with the complete Channelformer. We also propose an effective online training method based on the fifth generation (5G) new radio (NR) configuration for the modern communication systems, which only needs the available information at the receiver for online training. Using industrial standard channel models, the simulations of attention-based solutions show superior estimation performance compared with other candidate neural network methods for channel estimation.