# Learning Linear Block Error Correction Codes

arXiv: https://arxiv.org/abs/2405.04050 · HTML: https://mlsanity.com/paper/2405.04050/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [On Layer Normalization in the Transformer Architecture](https://mlsanity.com/paper/2002.04745/) (2020)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [OpenNMT: Open-source Toolkit for Neural Machine Translation](https://mlsanity.com/paper/1709.03815/) (2017)
- [Sionna: An Open-Source Library for Next-Generation Physical Layer Research](https://mlsanity.com/paper/2203.11854/) (2022)
- [Learning to Decode Linear Codes Using Deep Learning](https://mlsanity.com/paper/1607.04793/) (2016)
- [Error Correction Code Transformer](https://mlsanity.com/paper/2203.14966/) (2022)
- [Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels](https://mlsanity.com/paper/1911.03038/) (2019)
- [Communication Algorithms via Deep Learning](https://mlsanity.com/paper/1805.09317/) (2018)
- [Deepcode: Feedback Codes via Deep Learning](https://mlsanity.com/paper/1807.00801/) (2018)
- [Hyper-Graph-Network Decoders for Block Codes](https://mlsanity.com/paper/1909.09036/) (2019)
- [Deep Learning for Decoding of Linear Codes - A Syndrome-Based Approach](https://mlsanity.com/paper/1802.04741/) (2018)
- [Denoising Diffusion Error Correction Codes](https://mlsanity.com/paper/2209.13533/) (2022)
- [End-to-End Learning of Communications Systems Without a Channel Model](https://mlsanity.com/paper/1804.02276/) (2018)
- [Autoregressive Belief Propagation for Decoding Block Codes](https://mlsanity.com/paper/2103.11780/) (2021)
- [Deep Quantum Error Correction](https://mlsanity.com/paper/2301.11930/) (2023)
- [Graph Neural Networks for Channel Decoding](https://mlsanity.com/paper/2207.14742/) (2022)
- [Boosting Learning for LDPC Codes to Improve the Error-Floor Performance](https://mlsanity.com/paper/2310.07194/) (2023)
- [DeepTurbo: Deep Turbo Decoder](https://mlsanity.com/paper/1903.02295/) (2019)
- [How to Mask in Error Correction Code Transformer: Systematic and Double Masking](https://mlsanity.com/paper/2308.08128/) (2023)
- [perm2vec: Graph Permutation Selection for Decoding of Error Correction Codes using Self-Attention](https://mlsanity.com/paper/2002.02315/) (2020)

## Built on by (most cited first, 9 of 9)

- [Hybrid Mamba-Transformer Decoder for Error-Correcting Codes](https://mlsanity.com/paper/2505.17834/) (2025)
- [Factor Graph Optimization of Error-Correcting Codes for Belief Propagation Decoding](https://mlsanity.com/paper/2406.12900/) (2024)
- [Accelerating Error Correction Code Transformers](https://mlsanity.com/paper/2410.05911/) (2024)
- [Generalization Bounds for Transformer Channel Decoders](https://mlsanity.com/paper/2601.06969/) (2026)
- [Syndrome-Flow Consistency Model Achieves One-step Denoising Error Correction Codes](https://mlsanity.com/paper/2512.01389/) (2025)
- [Disturbance-based Discretization, Differentiable IDS Channel, and an IDS-Correcting Code for DNA-based Storage](https://mlsanity.com/paper/2407.18929/) (2024)
- [VT-Former: Efffcient Transformer-based Decoder for Varshamov-Tenengolts Codes](https://mlsanity.com/paper/2502.21060/) (2025)
- [Learning Linear Block Codes with Gradient Quantization](https://mlsanity.com/paper/2503.16169/) (2025)
- [Score Based Error Correcting Code Decoder](https://mlsanity.com/paper/2605.28358/) (2026)

## Abstract (arXiv)

Error correction codes are a crucial part of the physical communication layer, ensuring the reliable transfer of data over noisy channels. The design of optimal linear block codes capable of being efficiently decoded is of major concern, especially for short block lengths. While neural decoders have recently demonstrated their advantage over classical decoding techniques, the neural design of the codes remains a challenge. In this work, we propose for the first time a unified encoder-decoder training of binary linear block codes. To this end, we adapt the coding setting to support efficient and differentiable training of the code for end-to-end optimization over the order two Galois field. We also propose a novel Transformer model in which the self-attention masking is performed in a differentiable fashion for the efficient backpropagation of the code gradient. Our results show that (i) the proposed decoder outperforms existing neural decoding on conventional codes, (ii) the suggested framework generates codes that outperform the {analogous} conventional codes, and (iii) the codes we developed not only excel with our decoder but also show enhanced performance with traditional decoding techniques.