# Unified Error Correction Code Transformer with Low Complexity

arXiv: https://arxiv.org/abs/2410.03364 · HTML: https://mlsanity.com/paper/2410.03364/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 53 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting](https://mlsanity.com/paper/2012.07436/) (2020)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [Scatterbrain: Unifying Sparse and Low-rank Attention Approximation](https://mlsanity.com/paper/2110.15343/) (2021)
- [A3: Accelerating Attention Mechanisms in Neural Networks with Approximation](https://mlsanity.com/paper/2002.10941/) (2020)
- [Lessons on Parameter Sharing across Layers in Transformers](https://mlsanity.com/paper/2104.06022/) (2021)
- [Error Correction Code Transformer](https://mlsanity.com/paper/2203.14966/) (2022)
- [Hyper-Graph-Network Decoders for Block Codes](https://mlsanity.com/paper/1909.09036/) (2019)
- [The Road Towards 6G: A Comprehensive Survey](https://mlsanity.com/paper/2102.01420/) (2021)
- [Deep Learning for Decoding of Linear Codes - A Syndrome-Based Approach](https://mlsanity.com/paper/1802.04741/) (2018)
- [Massive MIMO is a Reality -- What is Next? Five Promising Research Directions for Antenna Arrays](https://mlsanity.com/paper/1902.07678/) (2019)
- [Autoregressive Belief Propagation for Decoding Block Codes](https://mlsanity.com/paper/2103.11780/) (2021)
- [Spectrum Breathing: Protecting Over-the-Air Federated Learning Against Interference](https://mlsanity.com/paper/2305.05933/) (2023)
- [Optimal Bilinear Equalizer for Cell-Free Massive MIMO Systems over Correlated Rician Channels](https://mlsanity.com/paper/2407.18531/) (2024)

## Built on by (most cited first, 1 of 1)

- [On the Optimality of Single-label and Multi-label Neural Network Decoders](https://mlsanity.com/paper/2503.18758/) (2025)

## Abstract (arXiv)

Channel coding is vital for reliable sixth-generation (6G) data transmission, employing diverse error correction codes for various application scenarios. Traditional decoders require dedicated hardware for each code, leading to high hardware costs. Recently, artificial intelligence (AI)-driven approaches, such as the error correction code Transformer (ECCT) and its enhanced version, the foundation error correction code Transformer (FECCT), have been proposed to reduce the hardware cost by leveraging the Transformer to decode multiple codes. However, their excessively high computational complexity of O(N2) due to the self-attention mechanism in the Transformer limits scalability, where N represents the sequence length. To reduce computational complexity, we propose a unified Transformer-based decoder that handles multiple linear block codes within a single framework. Specifically, a standardized unit is employed to align code length and code rate across different code types, while a redesigned low-rank unified attention module, with computational complexity of O(N), is shared across various heads in the Transformer. Additionally, a sparse mask, derived from the parity-check matrix's sparsity, is introduced to enhance the decoder's ability to capture inherent constraints between information and parity-check bits, improving decoding accuracy and further reducing computational complexity by 86%. Extensive experimental results demonstrate that the proposed unified Transformer-based decoder outperforms existing methods and provides a high-performance, low-complexity solution for next-generation wireless communication systems.